JP6062487B2 - Electronic device, method and program - Google Patents

Description

  Embodiments described herein relate generally to a technique for inputting a character string by handwriting.

  In recent years, various electronic devices such as tablets, PDAs, and smartphones have been developed. Many electronic devices of this type are equipped with a touch screen display to facilitate an input operation by a user.

  The user can instruct the electronic device to execute a function associated with the menu or object by touching the menu or object displayed on the touch screen display with a finger or the like.

  An input operation using a touch screen display is used not only to give an operation instruction to an electronic device but also to input a document by handwriting. Recently, handheld documents such as memos can be created by bringing this type of electronic device to attend meetings and performing handwritten input on a touch screen display.

  Further, in such an electronic device, a handwritten document created as described above by handwriting input a character string (hereinafter referred to as a search character string) as a search query (search key) on a touch screen display. Some of them can be searched.

  However, it takes effort to input all search character strings (for example, words) by handwriting.

JP 2007-265171 A JP 2005-025566 A

  An object of one embodiment of the present invention is to provide an electronic device, a method, and a program capable of displaying a character string candidate when a character string is input by handwriting.

According to an embodiment, the method is a region for inputting a search key for searching the handwritten document, and inputting the stroke data corresponding to a stroke set to be described by hand, n (n is 1 If the first stroke data corresponding to an integer greater than one) th stroke is input to the region is retrieved from the stroke set from first included in the handwritten document is a search target to n-th inputs and displaying the first candidate character string on the screen, if the second stroke data corresponding to n + 1-th stroke is input to the region, n + 1 from the first included in the handwritten document is a search target It is retrieved from the stroke set to th, comprising and displaying a second candidate character string to be input on the screen, before Kite writing document object stroke data It includes time-series information indicating the order relationship between the coordinate series and the stroke of the locus of each stroke of the stroke set corresponding to the stroke data entered at the time of the input, the first character string to the input candidate and second candidate, before Kite writing from the first on the basis of the time-series information contained in the document from the stroke set and the first to n-th search character string corresponding to the stroke set up n + 1 th When the first candidate of the input character string is different from the second candidate of the input character string after the second stroke data is input and displayed on the screen as a candidate, the input character string The second candidate of the input character string is preferentially displayed on the screen over the first candidate.

FIG. 1 is a perspective view illustrating an example of an external appearance of an electronic apparatus according to an embodiment. FIG. 2 is a block diagram illustrating an example of a cooperative operation between the electronic device and another device. FIG. 3 is a diagram illustrating an example of a handwritten document handwritten on the touch screen display. FIG. 4 is a diagram illustrating an example of time-series information that is a set of stroke data. FIG. 5 is a block diagram illustrating an example of a system configuration of the electronic device. FIG. 6 is a diagram illustrating a home screen displayed by the electronic device. FIG. 7 is a diagram illustrating a note preview screen displayed by the electronic device. FIG. 8 is a diagram illustrating a setting screen displayed by the electronic device. FIG. 9 is a diagram illustrating a page editing screen displayed by the electronic device. FIG. 10 is a diagram illustrating a search dialog displayed by the electronic device. FIG. 11 is a block diagram illustrating a functional configuration of a handwritten note application program executed by the electronic device. FIG. 12 is a diagram illustrating an example of the data structure of the suggest feature table. FIG. 13 is a diagram illustrating an example of the data structure of the suggest keyword table. FIG. 14 is a flowchart illustrating an example of the feature amount registration process. FIG. 15 is a diagram for specifically explaining the integrated character recognition processing. FIG. 16 is a flowchart illustrating an example of the candidate presentation process. FIG. 17 is a diagram for explaining keyword ranking. FIG. 18 is a diagram for explaining in detail a search dialog when displaying search character string candidates. FIG. 19 is a diagram for explaining in detail a search dialog when displaying search character string candidates. FIG. 20 is a diagram illustrating an example of a search result when the handwritten character string “apple” is selected as a search character string candidate.

Hereinafter, embodiments will be described with reference to the drawings.
FIG. 1 is a perspective view illustrating an example of an external appearance of an electronic apparatus according to an embodiment. This electronic device is, for example, a pen-based portable electronic device that can be handwritten with a pen or a finger. This electronic device can be realized as a tablet computer, a notebook personal computer, a smartphone, a PDA, or the like. Below, the case where this electronic device is implement | achieved as the tablet computer 10 is demonstrated. The tablet computer 10 is a portable electronic device also called a tablet or a straight computer, and the main body 11 has a thin box-shaped housing.

  The touch screen display 17 is attached to be superposed on the upper surface of the main body 11. The touch screen display 17 incorporates a flat panel display and a sensor configured to detect a contact position of a pen or a finger on the screen of the flat panel display. The flat panel display may be, for example, a liquid crystal display (LCD). As the sensor, for example, a capacitive touch panel, an electromagnetic induction digitizer, or the like can be used. Hereinafter, a case will be described in which both two types of sensors, the digitizer and the touch panel, are incorporated in the touch screen display 17. For this reason, the touch screen display 17 can detect not only a touch operation on the screen using a finger but also a touch operation on the screen using the pen 100.

  The pen 100 may be, for example, a digitizer pen (electromagnetic induction pen). The user can perform a handwriting input operation on the touch screen display 17 using the pen 100 (pen input mode). In the pen input mode, a trajectory of the movement of the pen 100 on the screen, that is, a stroke handwritten by a handwriting input operation is obtained, and a plurality of strokes input by handwriting are thereby displayed on the screen. The locus of movement of the pen 100 while the pen 100 is in contact with the screen corresponds to one stroke. A plurality of strokes constitute characters, symbols, and the like. A set of many strokes corresponding to handwritten characters, handwritten graphics, handwritten tables, and the like constitute a handwritten document.

  In the embodiment, the handwritten document is stored in the storage medium as time series information (handwritten document data) indicating the order relationship between the coordinate sequence of the trajectory of each stroke and the stroke, instead of image data. Details of the time-series information will be described later with reference to FIG. 4. The time-series information indicates the order in which a plurality of strokes are handwritten, and includes a plurality of stroke data respectively corresponding to the plurality of strokes. In other words, the time series information means a set of time series stroke data respectively corresponding to a plurality of strokes. Each stroke data corresponds to a certain stroke, and includes a coordinate data series (time series coordinates) corresponding to each point on the locus of this stroke. The order of arrangement of the stroke data corresponds to the order in which the strokes are handwritten.

  The tablet computer 10 can read existing arbitrary time-series information from the storage medium and display a handwritten document corresponding to the time-series information, that is, a plurality of strokes indicated by the time-series information, on the screen. The plurality of strokes indicated by the time series information are also a plurality of strokes input by handwriting.

  Furthermore, the tablet computer 10 according to the present embodiment also has a touch input mode for performing a handwriting input operation with a finger without using the pen 100. When the touch input mode is valid, the user can perform a handwriting input operation on the touch screen display 17 using a finger. In the touch input mode, a trajectory of finger movement on the screen, that is, a stroke handwritten by a handwriting input operation is obtained, and thereby a plurality of strokes input by handwriting are displayed on the screen.

  The tablet computer 10 has an editing function. This editing function is an arbitrary handwritten portion in a displayed handwritten document selected by the range selection tool in response to an editing operation by the user using the “eraser” tool, the range selection tool, and other various tools ( Handwritten characters, handwritten marks, handwritten graphics, handwritten tables, etc.) can be deleted or moved. Also, an arbitrary handwritten part in the handwritten document selected by the range selection tool can be designated as a search key for searching for a handwritten document. In addition, recognition processing such as handwritten character recognition / handwritten figure recognition / handwritten table recognition can be performed on an arbitrary handwritten portion in a handwritten document selected by the range selection tool.

  In the present embodiment, the handwritten document can be managed as one or a plurality of pages. In this case, a group of time-series information that fits on one screen may be recorded as one page by dividing time-series information (handwritten document data) by area units that fit on one screen. Alternatively, the page size may be variable. In this case, since the page size can be expanded to an area larger than the size of one screen, a handwritten document having an area larger than the screen size can be handled as one page. When one entire page cannot be displayed simultaneously on the display, the page may be reduced, or the display target portion in the page may be moved by vertical and horizontal scrolling.

  FIG. 2 shows an example of the linking operation between the tablet computer 10 and an external device. The tablet computer 10 includes a wireless communication device such as a wireless LAN, and can execute wireless communication with the personal computer 1. Furthermore, the tablet computer 10 can execute communication with the server 2 on the Internet 3 using a wireless communication device. The server 2 may be a server that executes an online storage service and other various cloud computing services.

  The personal computer 1 includes a storage device such as a hard disk drive (HDD). The tablet computer 10 can transmit time series information (handwritten document data) to the personal computer 1 and record it in the HDD of the personal computer 1 (upload). In order to secure secure communication between the tablet computer 10 and the personal computer 1, the personal computer 1 may authenticate the tablet computer 10 at the start of communication. In this case, a dialog prompting the user to input an ID or password may be displayed on the screen of the tablet computer 10, and the ID of the tablet computer 10 or the like is automatically transmitted from the tablet computer 10 to the personal computer 1. May be.

  Thereby, even when the storage capacity of the tablet computer 10 is small, the tablet computer 10 can handle a large amount of time-series information or a large amount of time-series information.

  Furthermore, the tablet computer 10 reads (downloads) any one or more time-series information recorded in the HDD of the personal computer 1 and displays the stroke indicated by the read time-series information on the screen of the display 17 of the tablet computer 10. Can be displayed. In this case, a list of thumbnails obtained by reducing each page of the plurality of pieces of time-series information may be displayed on the screen of the display 17, or one page selected from these thumbnails may be displayed on the screen of the display 17. You may display with normal size.

  Furthermore, the destination to which the tablet computer 10 communicates may be the server 2 on the cloud that provides a storage service or the like, as described above, instead of the personal computer 1. The tablet computer 10 can transmit time series information (handwritten document data) to the server 2 via the Internet and record it in the storage device 2A of the server 2 (upload). Further, the tablet computer 10 reads (downloads) arbitrary time-series information recorded in the storage device 2A of the server 2, and displays the trajectory of each stroke indicated by the time-series information on the screen of the display 17 of the tablet computer 10. Can be displayed.

  As described above, in this embodiment, the storage medium in which the time series information is stored may be any one of the storage device in the tablet computer 10, the storage device in the personal computer 1, and the storage device in the server 2.

  Next, with reference to FIG. 3 and FIG. 4, the relationship between strokes (characters, figures, tables, etc.) handwritten by the user and time-series information will be described. FIG. 3 shows an example of a handwritten document (handwritten character string) handwritten on the touch screen display 17 using the pen 100 or the like.

  In a handwritten document, in many cases, another character or figure is input by handwriting on the character or figure once input by handwriting. In FIG. 3, the handwritten characters “A”, “B”, and “C” are input by handwriting in this order, and then a handwritten arrow is input by handwriting in the immediate vicinity of the handwritten character “A”. ing.

  The handwritten character “A” is expressed by two strokes (“∧” shape trajectory, “−” shape trajectory) handwritten using the pen 100 or the like, that is, by two trajectories. The trajectory of the first “∧” -shaped pen 100 handwritten is sampled in real time, for example, at equal time intervals, thereby obtaining the time-series coordinates SD11, SD12,. Similarly, the trajectory of the “−” shaped pen 100 to be handwritten next is also sampled in real time at equal time intervals, thereby obtaining the time series coordinates SD21, SD22,..., SD2n of the “−” shaped stroke.

  The handwritten character “B” is represented by two strokes handwritten using the pen 100 or the like, that is, two trajectories. The handwritten character “C” is represented by one stroke handwritten using the pen 100 or the like, that is, one locus. The handwritten “arrow” is expressed by two strokes handwritten using the pen 100 or the like, that is, two trajectories.

  FIG. 4 shows time-series information 200 corresponding to the handwritten document of FIG. The time series information includes a plurality of stroke data SD1, SD2,. In the time series information 200, these stroke data SD1, SD2,... SD7 are arranged in time series in the order in which these strokes are handwritten.

  In the time series information 200, the first two stroke data SD1 and SD2 indicate two strokes of the handwritten character “A”, respectively. The third and fourth stroke data SD3 and SD4 indicate two strokes constituting the handwritten character “B”, respectively. The fifth stroke data SD5 indicates one stroke constituting the handwritten character “C”. The sixth and seventh stroke data SD6 and SD7 indicate two strokes constituting the handwritten “arrow”, respectively.

  Each stroke data includes a coordinate data series (time series coordinates) corresponding to one stroke, that is, a plurality of coordinates corresponding to a plurality of sampling points on the trajectory of one stroke. In each stroke data, the coordinates of a plurality of sampling points are arranged in chronological order in the order in which the strokes are written (sampled order). For example, for the handwritten character “A”, the stroke data SD1 is a coordinate data series (time series coordinates) corresponding to each point on the locus of the stroke of the “∧” shape of the handwritten character “A”, that is, n coordinates. Data SD11, SD12,..., SD1n are included. The stroke data SD2 includes coordinate data series corresponding to each point on the trajectory of the stroke of the “−” shape of the handwritten character “A”, that is, n pieces of coordinate data SD21, SD22,. Note that the number of coordinate data may be different for each stroke data. If the strokes are sampled at equal time intervals, the lengths of the strokes are different, so the number of sampling points is also different.

  Each coordinate data indicates the X coordinate and Y coordinate of a certain point in the corresponding locus. For example, the coordinate data SD11 indicates the X coordinate (X11) and the Y coordinate (Y11) of the start point of the “∧” -shaped stroke. SDn1 indicates the X coordinate (X1n) and Y coordinate (Y1n) of the end point of the “∧” -shaped stroke.

  Each coordinate data may include time stamp information T corresponding to the time point (sampling timing) when the point corresponding to the coordinates is handwritten. The handwritten time may be either absolute time (for example, year / month / day / hour / minute / second) or relative time based on a certain time. For example, absolute time (for example, year / month / day / hour / minute / second) at which the stroke is started is added to each stroke data as time stamp information, and each coordinate data in the stroke data indicates a difference from the absolute time. The relative time may be added as time stamp information T.

  As described above, by using the time series information in which the time stamp information T is added to each coordinate data, the temporal relationship between the strokes can be expressed more accurately. Although not shown in FIG. 4, information (Z) indicating writing pressure may be added to each coordinate data.

  The time series information 200 having the structure as described in FIG. 4 can represent not only the handwriting of each stroke but also the temporal relationship between the strokes. Therefore, by using this time-series information 200, as shown in FIG. 3, the tip of the handwritten “arrow” is written over the handwritten character “A” or close to the handwritten character “A”. However, the handwritten character “A” and the tip of the handwritten “arrow” can be handled as different characters or figures.

  Furthermore, in the present embodiment, as described above, handwritten document data is not stored as image or character recognition results, but is stored as time-series information 200 composed of a set of time-series stroke data. It can handle handwritten characters without depending on. Therefore, the structure of the time-series information 200 in this embodiment can be commonly used in various countries around the world with different languages.

  FIG. 5 is a diagram showing a system configuration of the tablet computer 10.

  The tablet computer 10 includes a CPU 101, a system controller 102, a main memory 103, a graphics controller 104, a BIOS-ROM 105, a nonvolatile memory 106, a wireless communication device 107, an embedded controller (EC) 108, and the like.

  The CPU 101 is a processor that controls the operation of various modules in the tablet computer 10. The CPU 101 executes various software loaded into the main memory 103 from the nonvolatile memory 106 that is a storage device. These software include an operating system (OS) 201 and various application programs. The various application programs include a handwritten note application program 202. Hereinafter, the handwritten document data is also referred to as a handwritten note. The handwritten note application program 202 has a function for creating and displaying the above handwritten document data, a function for editing the handwritten document data, handwritten document data including a desired handwritten part, and a desired handwritten part in a certain handwritten document data. It has a handwritten document search function for searching.

  The CPU 101 also executes a basic input / output system (BIOS) stored in the BIOS-ROM 105. The BIOS is a program for hardware control.

  The system controller 102 is a device that connects between the local bus of the CPU 101 and various component modules. The system controller 102 also includes a memory controller that controls access to the main memory 103. The system controller 102 also has a function of executing communication with the graphics controller 104 via a PCI EXPRESS serial bus or the like.

  The graphics controller 104 is a display controller that controls the LCD 17 </ b> A used as a display monitor of the tablet computer 10. A display signal generated by the graphics controller 104 is sent to the LCD 17A. The LCD 17A displays a screen image based on the display signal. Touch panel 17B, LCD 17A, and digitizer 17C are overlaid on each other. The touch panel 17B is a capacitance-type pointing device for inputting on the screen of the LCD 17A. The touch position on the screen where the finger is touched, the movement of the touch position, and the like are detected by the touch panel 17B. The digitizer 17C is an electromagnetic induction type pointing device for inputting on the screen of the LCD 17A. The contact position on the screen where the pen (digitizer pen) 100 is touched, the movement of the contact position, and the like are detected by the digitizer 17C.

  The wireless communication device 107 is a device configured to execute wireless communication such as wireless LAN or 3G mobile communication. The EC 108 is a one-chip microcomputer including an embedded controller for power management. The EC 108 has a function of powering on or off the tablet computer 10 in accordance with the operation of the power button by the user.

  Next, examples of some typical screens presented to the user by the handwritten note application program 202 will be described.

  FIG. 6 shows an example of the home screen of the handwritten note application program 202. The home screen is a basic screen for handling a plurality of handwritten document data, and can manage notes and set the entire application.

  The home screen includes a desktop screen area 70 and a drawer screen area 71. The desktop screen area 70 is a temporary area for displaying a plurality of note icons 801 to 805 corresponding to a plurality of handwritten notes being worked. Each of the note icons 801 to 805 displays a thumbnail of a page in the corresponding handwritten note. The desktop screen area 70 further displays a pen icon 771, a calendar icon 772, a scrap note (gallery) icon 773, and a tag (label) icon 774.

  The pen icon 771 is a graphical user interface (GUI) for switching the display screen from the home screen to the page editing screen. The calendar icon 772 is an icon indicating the current date. The scrap note icon 773 is a GUI for browsing data (scrap data or gallery data) imported from another application program or from an external file. The tag icon 774 is a GUI for attaching a label (tag) to an arbitrary page in an arbitrary handwritten note.

  The drawer screen area 71 is a display area for browsing a storage area for storing all created handwritten notes. The drawer screen area 71 displays note icons 80A, 80B, and 80C corresponding to some handwritten notes in all handwritten notes. Each of the note icons 80A, 80B, 80C displays a thumbnail of a page in the corresponding handwritten note. The handwriting note application program 202 can detect a certain gesture (for example, a swipe gesture) on the drawer screen area 71 performed by the user using the pen 100 or a finger. In response to detection of this gesture (for example, a swipe gesture), the handwritten note application program 202 scrolls the screen image on the drawer screen area 71 leftward or rightward. Thereby, the note icon corresponding to each arbitrary handwritten note can be displayed in the drawer screen area 71.

  The handwritten note application program 202 can detect another gesture (for example, a tap gesture) on the note icon in the drawer screen area 71 performed by the user using the pen 100 or a finger. In response to detection of a gesture on a note icon on the drawer screen area 71 (for example, a tap gesture), the handwritten note application program 202 moves the note icon to the center of the desktop screen area 70. Then, the handwritten note application program 202 selects a handwritten note corresponding to the note icon, and displays a note preview screen shown in FIG. 7 instead of the desktop screen. The note preview screen shown in FIG. 7 is a screen on which an arbitrary page in the selected handwritten note can be viewed.

  Furthermore, the handwritten note application program 202 can also detect a gesture (for example, a tap gesture) on the desktop screen area 70 performed by the user using the pen 100 or a finger. In response to detecting a gesture (for example, a tap gesture) on a note icon located at the center of the desktop screen area 70, the handwriting note application program 202 selects a handwritten note corresponding to the note icon located at the center. Then, a note preview screen shown in FIG. 7 is displayed instead of the desktop screen.

  Further, the home screen can display a menu. This menu includes a note list button 81A, a note creation button 81B, a note deletion button 81C, a search button 81D, and a setting button 81E displayed in the lower part of the screen, for example, the drawer screen area 71. The note list button 81A is a button for displaying a list of handwritten notes. The note creation button 81B is a button for creating (adding) a new handwritten note. The note deletion button 81C is a button for deleting a handwritten note. The search button 81D is a button for opening a search screen (search dialog). The setting button 81E is a button for opening an application setting screen.

  Although not shown, a return button, a home button, and a recent application button are also displayed below the drawer screen area 71.

  FIG. 8 shows an example of a setting screen that is opened when the setting button 81E is tapped with the pen 100 or a finger.

  This setting screen displays various setting items. These setting items include “backup and restoration”, “input mode (pen or touch input mode)”, “license information”, “help”, and the like.

  When note creation button 81B is tapped with pen 100 or a finger on the home screen, a note creation screen is displayed. Here, the name of the note is input by handwriting in the title field. Note that a notebook cover and paper can be selected. When the creation button is pressed, a new note is created, and the created note is placed in the drawer screen area 71.

  FIG. 7 shows an example of the above-described note preview screen.

  The note preview screen is a screen capable of browsing an arbitrary page in the selected handwritten note. Here, a case where a handwritten note corresponding to the note icon 801 in the desktop screen area 70 of the home screen is selected will be described. In this case, the handwritten note application program 202 can visually recognize a plurality of pages 901, 902, 903, 904 and 905 included in the handwritten note, at least a part of each of these pages 901, 902, 903, 904 and 905, In addition, these pages 901, 902, 903, 904, and 905 are displayed in an overlapping form.

  The note preview screen further displays the pen icon 711, the calendar icon 772, and the scrap note icon 773 described above.

  The note preview screen can further display a menu at the bottom of the screen. This menu includes a home button 82A, a page list button 82B, a page add button 82C, a page edit button 82D, a page delete button 82E, a label button 82F, a search button 82G, and a property display button 82H. The home button 82A is a button for closing the preview of the note and displaying the home screen. The page list button 82B is a button for displaying a list of pages in the currently selected handwritten note. The page addition button 82C is a button for creating (adding) a new page. The edit button 82D is a button for displaying a page edit screen. The page deletion button 82E is a button for deleting a page. The label button 82F is a button for displaying a list of types of labels that can be used. The search button 82G is a button for displaying a search screen. The property display button 82H is a button for displaying the property of this note.

  The handwritten note application program 202 can detect various gestures on the note preview screen performed by the user. For example, in response to detection of a certain gesture, the handwritten note application program 202 changes the page to be displayed at the top to an arbitrary page (page advance, page return). Also, in response to detection of a certain gesture (for example, tap gesture) performed on the top page, or in response to detection of a gesture (for example, tap gesture) performed on the pen icon 771 or editing. In response to detecting a gesture (eg, tap gesture) made on button 82D, handwriting note application program 202 selects the top page and instead of the note preview screen, the page shown in FIG. Display the edit screen.

  The page editing screen in FIG. 9 is a screen that allows a new creation of a page (handwritten page) in a handwritten note, and browsing and editing of an existing page. When the page 901 on the note preview screen in FIG. 7 is selected, the page edit screen displays the contents of the page 901 as shown in FIG.

  In this page editing screen, a rectangular area 500 surrounded by a broken line is a handwritten input area where handwriting can be performed. In the handwriting input area 500, an input event from the digitizer 17C is used for displaying (drawing) a handwritten stroke, and is not used as an event indicating a gesture such as a tap. On the other hand, in an area other than the handwriting input area 500 on the page editing screen, the input event from the digitizer 17C can be used even as an event indicating a gesture such as a tap.

  The input event from the touch panel 17B is not used for displaying (drawing) a handwritten stroke, but is used as an event indicating a gesture such as a tap or swipe.

  The page editing screen further displays a quick select menu including three types of pens 501 to 503 registered in advance by the user, a range selection pen 504, and an eraser pen 505 at the top of the screen outside the handwriting input area 500. Here, the case where the black pen 501, red pen 502, and marker 503 are registered in advance by the user is described. The user can switch the type of pen to be used by tapping a pen (button) in the quick select menu with the pen 100 or a finger. For example, when the handwriting input operation using the pen 100 is performed on the page editing screen in a state where the black pen 501 is selected by the user using the pen 100 or a tap gesture using a finger, the handwriting note application program 202 is A black stroke (trajectory) is displayed on the page edit screen in accordance with 100 movements.

  The above-mentioned three types of pens in the quick select menu can be switched by operating a side button (not shown) of the pen 100. For each of the above-described three types of pens in the quick select menu, combinations of frequently used pen colors and pen thicknesses can be set.

  The page editing screen further displays a menu button 511, a page return (return to note preview screen) button 512, and a new page addition button 513 at the bottom of the screen outside the handwriting input area 500. The menu button 511 is a button for displaying a menu.

  This menu includes, for example, putting this page in the trash, pasting a part of the copied or cut page, opening the search screen, displaying the export submenu, displaying the import submenu, converting the page to text Buttons for sending an e-mail and displaying a pen case may be displayed. The export submenu is, for example, a function for recognizing a handwritten page displayed on the page editing screen and converting it into an electronic document file, presentation file, image file, etc., or converting a page into an image file and other applications. Have users select the features to share. For example, the import submenu allows the user to select a function for importing a memo from the memo gallery or a function for importing an image from the gallery. The pen case is a button for calling up the pen setting screen that allows you to change the color (color of the line to be drawn) and thickness (thickness of the line to be drawn) of each of the three types of pens in the quick select menu. is there.

  FIG. 10 shows an example of a search screen (search dialog). FIG. 10 illustrates a case where the search button 82G is selected on the note preview screen shown in FIG. 7 and the search screen (search dialog) is opened on the note preview screen.

  The search screen displays a search key input area 530, a handwriting search button 531, a text search button 532, a delete button 533, and a search execution button 534. The handwriting search button 531 is a button for selecting handwriting search. The text search button 532 is a button for selecting text search. The search execution button 534 is a button for requesting execution of search processing.

  In handwriting search, the search key input area 530 is used as an input area for handwriting a character string (hereinafter referred to as a search character string), a figure, a table, and the like to be used as a search key. In FIG. 10, the handwritten character string “Dtermine” is input as a search character string in the search key input area 530. The user can handwrite not only the handwritten character string but also a handwritten figure, a handwritten table, and the like in the search key input area 530 with the pen 100. When the search execution button 434 is selected by the user in a state where the handwritten character string “Determine” is input as the search character string in the search key input area 530, the stroke set constituting the handwritten character string “Determine” as the search character string Using (query stroke set), a handwriting search for searching for a handwritten document (note) including a stroke set corresponding to the query stroke set is executed. In handwriting search, a stroke set similar to the query stroke set is searched by matching between strokes. In calculating the similarity between the query stroke set and another stroke set, DP (Dynamic Programming) matching may be used.

  In the present embodiment, when a search character string is input by handwriting in the search key input area 530, each time a stroke is handwritten by the user on the display, the search character string candidate is sent to the user. Presented. Details of the search screen (search dialog) relating to the presentation of search key candidates will be described later.

  On the other hand, in text search, for example, a software keyboard is displayed on the screen. The user can input an arbitrary text (character string) to the search key input area 530 as a search key by operating the software keyboard. When the search execution button 534 is selected by the user in a state where text is input as a search key in the search key input area 530, text search for searching for a handwritten note including a stroke set representing the text (query text) is performed. Executed.

  The handwriting search / text search can be executed for all handwritten documents, or can be executed only for selected handwritten documents. When handwriting search / text search is executed, a search result screen is displayed. On the search result screen, a list of handwritten documents (pages) including a stroke group corresponding to the query stroke group (or query text) is displayed. The hit word (stroke group corresponding to the query stroke group or query text) is highlighted.

  Next, the functional configuration of the handwritten note application program 202 will be described with reference to FIG.

  The handwritten note application program 202 is a WYSIWYG application capable of handling handwritten document data. The handwritten note application program 202 includes, for example, a display processing unit 301, a time-series information generation unit 302, a search processing unit 303, a page storage processing unit 304, a page acquisition processing unit 305, a feature amount registration processing unit 306, a work memory 401, and the like. Is provided. The display processing unit 301 includes a handwritten data input unit 301A, a handwriting drawing unit 301B, and a candidate presentation processing unit 301C.

  The touch panel 17B described above is configured to detect the occurrence of events such as “touch (contact)”, “movement (slide)”, and “release”. “Touch (contact)” is an event indicating that an object (finger) on the screen has touched. “Move (slide)” is an event indicating that the contact position has been moved while the object (finger) is in contact with the screen. “Release” is an event indicating that an object (finger) has been released from the screen.

  The digitizer 17C described above is also configured to detect the occurrence of events such as “touch (contact)”, “movement (slide)”, “release”, and the like. “Touch (contact)” is an event indicating that the object (pen 100) has touched the screen. “Move (slide)” is an event indicating that the contact position has been moved while the object (pen 100) is in contact with the screen. “Release” is an event indicating that the object (pen 100) has been released from the screen.

  The handwritten note application program 202 displays a page editing screen for creating, browsing, and editing handwritten page data on the touch screen display 17. The handwriting note application program 202 also taps the search button 81D on the home screen or the search button 82G on the note preview screen with the pen 100 or a finger to search the search screen (search dialog) described in FIG. Is displayed on the touch screen display 17.

  The display processing unit 301 and the time-series information generation unit 302 receive a “touch (contact)”, “movement (slide)”, or “release” event generated by the digitizer 17C, and thereby detect a handwriting input operation. . The “touch (contact)” event includes the coordinates of the contact position. The “movement (slide)” event includes the coordinates of the contact position of the movement destination. Therefore, the display processing unit 301 and the time-series information generation unit 302 can receive (input) a coordinate sequence (that is, stroke data) corresponding to the movement locus of the contact position from the digitizer 17C.

  The display processing unit 301 displays a handwritten stroke on the screen according to the movement of the object (pen 100) on the screen detected using the digitizer 17C. The display processing unit 301 displays the trajectory of the pen 100 while the pen 100 is in contact with the screen, that is, the trajectory of each stroke, on the page editing screen or the search screen (the search key input area 530 in the handwriting search described above). The

  The time series information generation unit 302 receives the above-described coordinate sequence output from the digitizer 17C, and based on this coordinate sequence, time-series information (coordinate data sequence) having the structure described in detail in FIG. Generate handwritten document data. Note that the handwritten document data generated by the time-series information generation unit 302 includes a time-series stroke set handwritten by the user on the display. The time series information generation unit 302 temporarily stores the generated handwritten document data in the work memory 401.

  The search processing unit 303 has a search index therein, and executes the above-described handwriting search and text search using the search index. Specifically, the search processing unit 303 performs an operation (search for a search execution button 534 to be tapped with the pen 100 or a finger after a search character string (search key) is input to the search key input area 530 on the search screen described above. When an instruction operation is performed, the page acquisition processing unit 307 is instructed to acquire a handwritten document (handwritten page) including a stroke group corresponding to the search character string as a search result.

  The page storage processing unit 304 stores handwritten document data including a plurality of stroke data corresponding to a plurality of handwritten strokes on a handwritten document (handwritten page) being created in the storage medium 402. The storage medium 402 may be, for example, a storage device in the tablet computer 10 or a storage device of the server computer 2.

  The page acquisition processing unit 305 acquires arbitrary handwritten document data (handwritten page data) from the storage medium 402. Specifically, the page acquisition processing unit 307 acquires a handwritten document based on the instruction from the search processing unit 303 described above. The acquired handwritten document data is sent to the display processing unit 301. The display processing unit 301 displays a plurality of strokes corresponding to the plurality of stroke data included in the handwritten document data sent from the page acquisition unit 305 on the screen.

  When the handwritten document (data) is stored in the storage medium 402 by the page storage processing unit 306, the feature amount registration processing unit 306 performs a character recognition process on the stroke set that constitutes the handwritten document. All strokes constituting the handwritten document are converted into character strings (words). The feature amount registration processing unit 306 uses the converted character string as a keyword, and each stroke in the stroke set converted into the keyword and the keyword in the handwritten document (that is, character recognition as the keyword by the character recognition process) is performed. A character recognition result for each stroke set obtained by integrating one stroke in time series order and the number of strokes in the stroke set are associated and registered in the suggest feature table. Further, the feature amount registration processing unit 306 registers the converted character string (keyword) and the stroke data corresponding to the stroke set converted to the character string in the suggestion keyword table. It is assumed that the suggest feature table and the suggest keyword table are stored in the storage medium 402, for example.

  Next, details of the display processing unit 301 illustrated in FIG. 11 will be described.

  As described above, the touch screen display 17 detects a touch operation on the screen by the touch panel 17B or the digitizer 17C. The handwritten data input unit 301A is a module that inputs a detection signal output from the touch panel 17B or the digitizer 17C. The detection signal includes coordinate information (X, Y) of the touch position. By inputting such detection signals in chronological order, the handwritten data input 301A inputs, for example, stroke data corresponding to a stroke described by handwriting on a display. The stroke data (detection signal) input by the handwritten data input unit 301A is supplied to the handwriting drawing unit 301B.

  The handwriting drawing unit 301 </ b> B is a module that draws a handwriting input locus (handwriting) and displays it on the LCD 17 </ b> A of the touch screen display 17. The handwriting drawing unit 301B draws a line segment corresponding to the locus (handwriting) of handwriting input based on the stroke data (detection signal) from the handwriting data input unit 301A.

  Here, when the stroke data input by the handwritten data input unit 301A corresponds to the stroke handwritten in the search key input area 530 on the search screen described above, the stroke data is supplied to the candidate presentation processing unit 301C. Is done. When stroke data corresponding to a stroke is input by the handwritten data input unit 301A in this way, the candidate presentation processing unit 301C has input stroke data (that is, the input data) when the stroke data is input. The search character string candidates corresponding to the stroke set corresponding to the stroke data and the stroke data already input) are displayed on the screen. At this time, the candidate presentation processing unit 301C displays the search character string candidates to the user via the LCD 17A with reference to the suggestion feature table and the suggestion keyword table stored in the storage medium 402 described above.

  In this way, when the search character string candidate displayed on the screen is selected by the user, the above-described search processing by the search processing unit 303 is executed based on the search character string (candidate).

  Although not shown in FIG. 11, the handwritten note application program 202 includes an editing processing unit that executes processing for editing a handwritten document currently displayed, in addition to the above. This editing processing unit adds new strokes (new handwritten characters, new handwritten marks, etc.) to the currently displayed handwritten document in response to editing operations and handwriting input operations performed by the user on the touch screen display 17. Edit processing including processing to delete, processing to delete or move one or more of the displayed strokes, and the like. Further, the editing processing unit updates the time series information in the work memory 401 in order to reflect the result of the editing process in the time series information being displayed.

  FIG. 12 shows an example of the data structure of the suggestion feature table stored in the storage medium 402 described above. As shown in FIG. 12, in the suggestion feature table, keywords, character recognition results, and the number of strokes are stored (registered) in association with each other. The keyword is a character string (word) corresponding to a search character string candidate. The character recognition result indicates a character recognition result for a part of a stroke set of a stroke set (handwritten character string) that is recognized as a keyword associated with the character recognition result. The number of strokes indicates the number of strokes in the stroke set from which the character recognition result associated with the number of strokes is obtained.

  In the example illustrated in FIG. 12, for example, the keyword “application”, the character recognition result “a”, and the number of strokes “1” are held in association in the suggestion feature table. According to this, when a stroke set recognized as the keyword “application” is handwritten by the user, the character recognition result when the character recognition process is performed when one stroke is handwritten is “a”. It has been shown.

  In the suggestion feature table, for example, the keyword “application”, the character recognition result “ap”, and the number of strokes “2” are stored in association with each other. According to this, when the stroke set recognized as the keyword “application” is handwritten by the user, the character recognition result when the character recognition process is performed when two strokes are handwritten is “ap”. It has been shown.

  In the example shown in FIG. 12, it is assumed that the character strings “a” and “p” are each handwritten with one stroke.

  As described above, the suggestion feature table holds the character recognition result every time the number of strokes (that is, the number of strokes) constituting the keyword “application” is incremented by one. That is, in the suggestion feature table, as described above, the character recognition result for each stroke set obtained by integrating the strokes in the stroke set recognized as a keyword one by one in time-series order and the number of strokes in the stroke set Stored in association with keywords.

  Although details will be described later, when the search character string candidates are displayed as described above, a search is performed using the character recognition result and the number of strokes as keys.

  FIG. 13 shows an example of the data structure of the suggest keyword table stored in the storage medium 402 described above. As shown in FIG. 13, in the suggestion keyword table, a key key keyword, stroke data, appearance frequency, and number of selections are stored (registered) in association with each other. The keyword is a character string corresponding to a search character string candidate. The stroke data is data (binary data of the stroke) corresponding to a stroke set that is recognized as a keyword associated with the stroke data. The appearance frequency indicates, for example, a frequency at which a keyword (a stroke set recognized as a character) associated with the appearance frequency appears in a handwritten document saved in the storage medium 402. As will be described later, when the search character string candidate is displayed on the screen based on the keyword and stroke data associated with the selection number, as described later, the search character string candidate (search character string) As) Indicates the number of times selected by the user.

  In the example illustrated in FIG. 13, for example, the keyword “app”, stroke data “(10, 10)-(13, 8)-...”, Appearance frequency “5”, and selection frequency “3” correspond to the suggestion keyword table. Is retained. This indicates that the stroke data corresponding to the stroke set recognized as the keyword “app” is “(10, 10)-(13, 8)-...”. Further, it is shown that the keyword “app” (as a stroke set recognized as a character) appears five times in the handwritten document saved in the storage medium 402. Further, when search character string candidates are displayed on the screen based on the keyword “app” and the stroke data “(10, 10)-(13, 8) —. Shows that it was selected three times. As described above, the stroke data includes a plurality of coordinates corresponding to a plurality of sampling points on the trajectory of one stroke.

  Although only the keyword “app” has been described here, the same applies to other keywords.

  Hereinafter, the operation of the tablet computer 10 according to the present embodiment will be described. Here, the above-described handwritten document (data) is described by handwriting in the processing of the feature amount registration processing unit 306 (hereinafter referred to as feature amount registration processing) and the search key input area 530 when it is stored in the storage medium 402. The processing (hereinafter referred to as candidate presentation processing) of the candidate presentation processing unit 301C when stroke data corresponding to a stroke is input will be mainly described.

  First, the processing procedure of the above-described feature amount registration processing will be described with reference to the flowchart of FIG.

  In the feature amount registration process, the feature amount registration processing unit 306 acquires the handwritten document from, for example, the work memory 401 when the page storage processing unit 305 stores the handwritten document in the storage medium 402 (step S1). The handwritten document is composed of a stroke set handwritten by the user in the handwriting input area of the page editing screen described above, and includes stroke data corresponding to the stroke set.

  Next, the feature amount registration processing unit 306 performs a character recognition process on the acquired handwritten document (a set of strokes corresponding to the stroke data included therein) (step S2). Thereby, the stroke set which comprises a handwritten document is converted into a character string. At this time, each stroke (corresponding to stroke data) constituting the handwritten document corresponds to a character (character constituting the stroke) to which the stroke belongs in the character string converted by executing the character recognition process. It shall be attached.

  The feature amount registration processing unit 306 performs a morpheme analysis (morpheme analysis) process on the converted character string (step S3). As a result, the converted character string is divided into words. At this time, the feature amount registration processing unit 306 specifies a stroke set belonging to each word divided by the above-described morpheme analysis processing based on the stroke associated with each character in the above-described character string. That is, here, each word divided by the morphological analysis process and the stroke arrangement of the word are specified.

  Next, the feature amount registration processing unit 306 performs integrated character recognition processing on the stroke set belonging to each word divided by the morpheme analysis processing (step S4). This integrated character recognition process is a process of acquiring a character recognition result (character string) that is a feature amount for each stroke.

  Here, with reference to FIG. 15, the integrated character recognition process will be described in detail. Here, a case where the integrated character recognition process is executed for the stroke set belonging to the word “apple” will be described.

  In this case, the character recognition result when the character recognition process is executed for the stroke (collection) 1001 having the number of strokes (number of strokes) of 1 is “a”.

  Next, the character recognition result when the character recognition process is executed for the stroke set 1002 with the number of strokes (number of strokes) is “ap”.

  Similarly, the character recognition result when the character recognition process is executed for the stroke set 1003 having the number of strokes (number of strokes) of 3 is “app”.

  Further, the character recognition result when the character recognition process is executed for the stroke set 1004 having the number of strokes (number of strokes) is “appl”.

  Finally, the character recognition result when the character recognition process is executed for the stroke set 1005 having the number of strokes (number of strokes) of 5 is “apple”.

  As described above, when the integrated character recognition process is executed for the stroke set belonging to the word “apple”, the integrated character recognition result 1100 shown in FIG. 15 can be obtained. The integrated character recognition result 1100 includes a word recognition result for a word and a stroke set and the number of strokes (the number of strokes in the stroke set).

  Returning to FIG. 14 again, the feature amount registration processing unit 306 registers various information in the above-described suggest feature table and suggest keyword table based on the acquired integrated character recognition result 1100 (step S5).

  Specifically, the feature amount registration processing unit 306 registers the word (keyword), the character recognition result, and the number of strokes included in the accumulated character recognition result 1100 in association with each other in the suggestion feature table. The feature amount registration processing unit 306 registers the word (keyword) included in the integrated character recognition result and the stroke data corresponding to the stroke set belonging to the word in the suggestion keyword table. In the suggestion keyword table, the frequency (appearance frequency) at which the keyword registered in the suggestion keyword table appears in the handwritten document stored in the storage medium 402 is further registered.

  If the same information is already held in the suggestion feature table and the suggestion keyword table in step S5, the information registration process is omitted.

  In addition to the above-described keywords, stroke data, and appearance frequency, the number of selections (information thereof) is also stored in the suggestion keyword table. In this feature amount registration process, 0 is registered as the number of selections. The number of selections held in the suggest keyword table is updated when a search character string candidate displayed by a candidate presentation process described later is selected by the user.

  As described above, according to the feature amount registration process, when a handwritten document is stored in the storage medium 402, necessary information used in a candidate presentation process to be described later is automatically registered in the suggest feature table and the suggest keyword table. be able to.

  Next, with reference to the flowchart of FIG. 16, the processing procedure of the above candidate presentation processing will be described. The candidate presentation process is executed each time a stroke constituting a search character string (character string to be used as a search key) is written by hand in the search key input area 530 on the search screen.

  First, the candidate presentation processing unit 301C inputs stroke data corresponding to one stroke described by handwriting by the user in the search key input area 530 on the display (step S11). Hereinafter, the stroke data input here is referred to as target stroke data.

  Next, the candidate presentation processing unit 301C performs the stroke set corresponding to the stroke data input at the time when the target stroke data is input (that is, the stroke set written in the search key input area 530 by handwriting). Character recognition processing (integrated character recognition processing) is executed (step S12). Specifically, if the target stroke data is stroke data corresponding to the stroke of the nth stroke of the search key, the candidate presentation processing unit 301C performs character recognition for the stroke set of the first stroke to the nth stroke. Execute the process. Thereby, candidate presentation processing part 301C acquires a character recognition result.

  Based on the acquired character recognition result and the number of strokes in the stroke set from which the character recognition result is acquired, the candidate presentation processing unit 301C responds to the stroke set (that is, the stroke set of the first stroke to the nth stroke). The keyword (that is, the search character string candidate) is searched (step S13). Specifically, the candidate presentation processing unit 301 </ b> C is a keyword stored in the suggestion feature table in association with the acquired character recognition result and the number of strokes (that is, the number of strokes) in the stroke set from which the character recognition result is acquired. Search for. In step S13, a plurality of keywords may be searched.

  Next, the candidate presentation processing unit 301C ranks the searched keywords (step S14).

  Here, keyword ranking will be described. In character recognition, there is a case where it cooperates with a language dictionary. For example, when character recognition processing is performed in the middle of handwritten input of a search character string (for example, character recognition processing is performed on a stroke set of “ap”), meaning May be converted to a close word of some form. For this reason, there is a case where the search character string intended by the user cannot be presented (displayed) in such a state that does not make sense during handwriting input. In general, as the number of characters increases, the possibility of being a correct candidate increases.

  Therefore, in the present embodiment, stroke data (second stroke) corresponding to the stroke of the nth stroke after the stroke data corresponding to the stroke of the (n-1) th stroke (first stroke data corresponding to the first stroke). When the second stroke data corresponding to the stroke is input and the stroke data corresponding to the stroke of the (n-1) th stroke is input, a set of stroke data already input (that is, the first stroke data) The search character string candidate (first character string candidate) corresponding to the stroke set of the (n-1) th stroke) and the stroke data already input when the stroke data corresponding to the stroke of the nth stroke are input. If the search character string candidate (second character string candidate) according to the set (that is, the stroke set of the first stroke to the nth stroke) is different, the first stroke to the n-1th stroke To rank as a candidate search character string corresponding to one screen th ~n image th stroke set than the candidate search character string corresponding to the stroke set is displayed preferentially to the screen.

  Here, the ranking of keywords will be described in detail with reference to FIG. Here, it is assumed that the user inputs “apple” as a search character string by handwriting. As shown in FIG. 17, the character recognition result for the stroke of the first stroke is “a”, the first stroke and the second stroke. The character recognition result for the eye stroke set is “as”, the character recognition result for the first to third stroke sets is “app”, and the character recognition result for the first to fourth stroke sets is “appl”. A case where the character recognition result for the stroke set of the first stroke to the fifth stroke is “apple” will be described.

  Note that the character recognition result for the stroke set of the first and second strokes is “as” instead of “ap”. For example, when the user should originally input “p” by handwriting, This is because a character that is recognized as “s” is input by handwriting in the second stroke. The character recognition result for the stroke set of the first to third strokes is not “asp” but “app” because the character recognition result for the stroke of the third stroke is “p”. The character recognition result for the stroke set of the third to third strokes is “asp”. For example, it is determined that the keyword starting with “asp” does not exist in the suggestion feature table shown in FIG. 12, the suggestion keyword table shown in FIG. This is because it was recognized (corrected) as “app”.

  First, when the stroke (data) of the first stroke is input, the character recognition result is “a”. Therefore, in the process of step S13 described above, for example, “apple”, “approve”, and “application” are used as keywords. Is searched. In this case, the number of strokes (here, 1) is given as a point for ranking to each of the searched keywords “apple”, “approve”, and “application”. In FIG. 17, the numerical value in [] represents a point given to each keyword.

  Next, when the stroke (data) of the second stroke is input, the character recognition result is “as”. Therefore, for example, “asterisk” is searched from the suggestion feature table as a keyword in the process of step S13 described above. The In this case, the number of strokes (here, 2) is given as a point for ranking to the searched keyword “asterisk”. Note that the points for the keywords “apple”, “approve”, and “application” that are not searched when the stroke of the second stroke is input are 1 (that is, maintained) as in the first stroke. .

  Next, when the stroke (data) of the third stroke is input, the character recognition result is “app”. Therefore, in the above-described step S13, for example, “apple”, “approve”, and “application” are keywords. Retrieved from the suggest feature table. In this case, the number of strokes (here, 3) is given as a point for ranking to the searched keywords “apple”, “approve”, and “application”. As described above, when points are given to the keywords “apple”, “approve”, and “application”, the points of the keywords “apple”, “approve”, and “application” are the points at the time of the first stroke. Total is 4 each. Note that the point for the keyword “asterisk” that has not been searched when the stroke of the third stroke is input is 1 (that is, maintained) as in the second stroke.

  Next, when the stroke (data) of the fourth stroke is input, the character recognition result is “appl”. Therefore, in the above-described step S13, for example, “apple” and “application” are used as keywords from the suggestion feature table. Searched. In this case, the number of strokes (here, 4) is given as a point for ranking to the searched keywords “apple” and “application”. When points are given to the keywords “apple” and “application” in this way, the points of the keywords “apple” and “application” total 8 points at the time of the third stroke. The points for the keywords “approve” and “asterisk” that are not searched when the stroke of the fourth stroke is input are 4 and 2, respectively (that is, maintained) as in the third stroke.

  Finally, when the stroke (data) of the fifth stroke is input, the character recognition result is “apple”. Therefore, for example, “apple” is retrieved from the suggestion feature table as a keyword in step S13 described above. In this case, the number of strokes (here, 5) is given as a point for ranking to the searched keyword “apple”. When points are given to the keyword “apple” in this way, the points of the keyword “apple” are 13 in total with the points at the time of the fourth stroke. Note that the points for the keywords “application”, “approve”, and “asterisk” that are not searched when the stroke of the fifth stroke is input are 8, 4 and 2, respectively, as in the fourth stroke.

  As described above, in this embodiment, for example, each time a stroke is handwritten so that n points are given to a keyword searched for by the stroke of the nth image (a keyword matched by the matching of the nth image). Is ranked so that keywords with high total point values (search character string candidates) are displayed at the top (that is, displayed in order of ranks 1 to 4).

  Returning to FIG. 16 again, the candidate presentation processing unit 301C obtains stroke data corresponding to the stroke set constituting the searched keyword (step S15). Specifically, the candidate presentation processing unit 301C acquires the stroke data held in the suggestion keyword table in association with the searched keyword.

  The candidate presentation processing unit 301C displays search character string candidates by drawing the searched keyword and the acquired stroke data on a display (screen) (step S16). In this case, the searched keyword is displayed as text, and the acquired stroke data is displayed as a handwritten character string.

  The search character string candidates are preferentially displayed on the screen in the descending order of the points given to the search character string candidates (keywords) in the above ranking.

  That is, in the example shown in FIG. 17, when the stroke (data) of the first stroke is input, “apple”, “approve”, and “application” (text and handwritten character string) as search character string candidates. Is displayed on the screen. When the points given to these search character string candidates (keywords) are the same, the search character string candidates may be displayed in an arbitrary order. The same applies to the second and subsequent strokes.

  When the stroke (data) of the second stroke is input, “asterisk”, “apple”, “approve”, and “application” (text and handwritten character string) are displayed on the screen as search character string candidates. Is displayed. Here, the points given to “asterisk” are higher than the points given to “apple”, “approve”, and “application”. In this case, “asterisk” is displayed preferentially over “apple”, “approve”, and “application”.

  If the stroke (data) of the third stroke is input, “apple”, “approve”, “application”, and “asterisk” (text and handwritten character string) are displayed on the screen as search character string candidates. Is displayed. Here, the points given to “apple”, “approve”, and “application” are higher than the points given to “asterisk”. In this case, “apple”, “approve”, and “application” are displayed with priority over “asterisk”.

  When the stroke (data) of the fourth stroke is input, “apple”, “application”, “approve”, and “asterisk” (text and handwritten character string) are displayed on the screen as search character string candidates. Is displayed. Here, the points given to “apple” and “application” are higher than the points given to “approve” and “asterisk”. In this case, “apple” and “application” are displayed with priority over “approve” and “asterisk”. Furthermore, the points given to “approve” are higher than the points given to “asterisk”. In this case, “approve” is displayed with priority over “asterisk”.

  If the stroke (data) of the fifth stroke is input, “apple”, “application”, “approve”, and “asterisk” (text and handwritten character string) are displayed on the screen as search character string candidates. Is displayed. Here, the points given to “apple” are higher than the points given to “application”, “approve”, and “asterisk”. In this case, “apple” is displayed with priority over “application”, “approve”, and “asterisk”. Furthermore, the points given to “application” are higher than the points given to “approve” and “asterisk”. In this case, “application” is displayed with priority over “approve” and “asterisk”. Similarly, the point given to “approve” is higher than the point given to “asterisk”. In this case, “approve” is displayed with priority over “asterisk”.

  Thus, in this embodiment, the search character string candidate can be changed each time a stroke is written by hand on the display.

  Here, the search character string candidate is described as being displayed by both text and handwritten character string. However, if the search character string candidate is displayed by at least one of text or handwritten character string, for example. Good.

  Also, here, when the points given to a plurality of search character string candidates are the same, the search character string candidates are described as being displayed on the screen in an arbitrary order. For example, FIG. The ranking may be further performed according to the appearance frequency held in the suggested keyword table. In this case, among the plurality of search character string candidates (keywords) to which the same point is given, search character string candidates with high appearance frequency are preferentially displayed on the screen.

  Further, the ranking may be further performed according to the number of selections held in the suggestion keyword table shown in FIG. In this case, among the plurality of search character string candidates to which the same point is assigned, search character string candidates with a large number of selections are displayed with priority.

  Note that the above-described appearance frequency and selection count (information) are not necessarily used, and when the appearance frequency and selection count are not used, the appearance frequency and selection count are not held in the suggestion keyword table. It can also be. On the other hand, when the appearance frequency and the number of selections are held in the suggestion keyword table, ranking is performed using both the appearance frequency and the number of selections, and ranking is performed using one of the appearance frequency and the number of selections. It is also possible to adopt a configuration in which the user selects either of the appearance frequency and the selection frequency. Furthermore, when ranking is performed using both the appearance frequency and the number of times of selection, it is possible to set which of the appearance frequency and the number of times of selection is prioritized.

  In the example shown in FIG. 17, when the stroke (data) of the fifth stroke is input, “apple”, “application”, “approve”, and “asterisk” are displayed on the screen as search character string candidates. Although described as being displayed, only a part of the search character string candidates may be displayed on the screen according to the points (priorities) given to the search character string candidates. Specifically, for example, only search character string candidates to which one-third or more points of the maximum value of points given to each of a plurality of search character string candidates are assigned are displayed on the screen. It can be set as a simple structure. In the case of such a configuration, when the stroke (data) of the fifth stroke shown in FIG. 17 is input, only “apple” and “application” can be displayed on the screen as search character string candidates. .

  Next, with reference to FIGS. 18 and 19, a search dialog for displaying search character string candidates will be described in detail. Here, the same parts as those in FIG. 10 described above are denoted by the same reference numerals, and detailed description thereof is omitted.

  FIG. 18 shows an example of a search dialog when a set of strokes recognized as “ap” in the character recognition process (first and second strokes) is handwritten in the search key input area 530. FIG. 19 shows an example of a search dialog when a stroke set recognized as “appl” in the character recognition process (strokes in the first to fourth strokes) is handwritten in the search key input area 530.

  As shown in FIGS. 18 and 19, the search dialog includes a candidate presentation area 535 and a presentation mode selection area 536.

  The candidate presentation area 535 is an area in which search character string candidates corresponding to the stroke set written by handwriting in the search key input area 530 are displayed. Specifically, as shown in FIGS. 18 and 19, a handwritten character string based on the stroke data acquired in step S <b> 15 shown in FIG. 16 described above is displayed on the upper stage of the candidate presentation area 535. On the other hand, below the candidate presentation area 535, text based on the keyword searched in step S13 shown in FIG. 16 described above is displayed. In FIG. 18 and FIG. 19, search character string candidates that are displayed preferentially (that is, search character string candidates with high points given for ranking) are displayed in order from the left side.

  In the example illustrated in FIG. 18, “apple”, “application”, and “approve” are sequentially displayed from the left side as candidates for the search character string in the candidate presentation area 535. According to this, for example, “apple” can be recognized as a search character string candidate that is displayed preferentially based on the points given for ranking and the appearance frequency.

  In the example shown in FIG. 19, “apple” and “application” are displayed in order from the left as candidates for the search character string in the candidate presentation area 535. According to this, for example, “apple” can be recognized as a search character string candidate that is displayed preferentially based on the points given for ranking and the appearance frequency. Further, in the example shown in FIG. 19, for example, only a part of the search character string candidates described above is displayed on the screen, and among the search character string candidates displayed in the candidate presentation area 535 shown in FIG. 18. , Candidates that do not include “pl” are not displayed.

  In the example illustrated in FIGS. 18 and 19, the search character string candidates that are preferentially displayed (that is, search character string candidates with high priority) are displayed in order from the left side. The search character string candidates displayed automatically may be displayed in other manners. Specifically, search character string candidates that are displayed preferentially may be displayed in a manner distinguishable from other search character string candidates, for example, by changing the display color.

  The presentation mode selection area 536 is an area for the user to select a search mode candidate presentation mode (“Stroke & Text”, “Stroke only”, and “Text only”). That is, in the presentation mode selection area 536, whether the search character string candidate is displayed as both the handwritten character string and the text, is displayed only as the handwritten character string, or is displayed only as the text. The user can select.

  For example, when “Stroke & Text” is selected by the user in the presentation mode selection area 536, the above-described candidate presentation area 535 includes a handwritten character string and a text as search character string candidates as shown in FIGS. Both are displayed. On the other hand, for example, when “Stroke only” is selected by the user in the presentation mode selection area 536, only the handwritten character string is displayed as a candidate for the search character string in the candidate presentation area 535 described above. When, for example, “Text only” is selected by the user in the presentation mode selection area 536, only the text is displayed as a candidate for the search character string in the candidate presentation area 535 described above.

  Further, the user can select (designate) a desired search character string from among the search character string candidates displayed on the search screens shown in FIGS. In this case, the search process is executed based on the search character string (candidate) selected by the user.

  Specifically, when a handwritten character string displayed as a search character string candidate is selected, the handwriting search described above is executed based on the handwritten character string. According to the handwriting search, a handwritten document including a stroke set corresponding to the query stroke set is searched using a stroke set (query stroke set) constituting the selected handwritten character string.

  On the other hand, when the text displayed as the search character string candidate is selected, the above-described text search is executed based on the text. According to this text search, a handwritten document including a stroke set representing the selected text is searched.

  Here, although it has been described that the search process is executed when a search character string candidate is selected by the user, when the search character string candidate is selected, it is temporarily selected in the search key input area 530 A configuration may be adopted in which search processing is executed when a search execution button 534 is selected by the user after a search character string candidate (for example, a handwritten character string) is displayed.

  Here, FIG. 20 shows an example of a search result when the handwritten character string “apple” is selected as a search character string candidate, for example. The search result when the handwritten character string “apple” is selected as a candidate for the search character string includes a handwritten document including a stroke set corresponding to the query stroke set constituting the handwritten character string “apple”. In the handwritten document included in the search result, as shown in FIG. 20, a stroke set corresponding to the query stroke set constituting the handwritten character string “apple” is highlighted.

  By executing the candidate presentation process described above every time a stroke is handwritten in the search key input area 530, the displayed search character string candidates can be updated.

  When the search character string candidate displayed on the search screen shown in FIGS. 18 and 19 is selected by the user as a desired search character string, it is used to display the selected search character string candidate. 1 is added to the number of selections held in the suggest keyword table in association with the keyword or stroke data. Thus, each time a search character string candidate is selected by the user, the number of times the search character string candidate is selected can be updated.

  As described above, in the present embodiment, when stroke data corresponding to a stroke written in handwriting is input and stroke data corresponding to the stroke is input, the stroke data is input at the time of input. With the configuration in which search character string candidates corresponding to the stroke set corresponding to the stroke data are displayed on the screen, the search character string can be input in fewer steps without handwriting all of the search character string (search key). Therefore, it is possible to save the labor of the search work. The search character string candidates are searched and displayed using the character recognition result for the stroke set written by hand on the display as described above and the number of strokes (number of strokes) in the stroke set as keys. Character string candidates (noise) can be reduced, and only appropriate search character string candidates can be displayed.

  Further, in the present embodiment, it is possible to perform a search only by selecting a candidate displayed on the screen without handwriting all of the search character string, so a word whose spelling or writing is unknown is searched. Even in the case of a character string, it is possible to perform a search with an appropriate search character string.

  Furthermore, in the present embodiment, since a one-dimensional quantity called a character recognition result for a stroke set described by handwriting is used, a lightweight search algorithm can be used as compared with the case of using a multi-dimensional quantity, and a search character string It is possible to speed up the processing when displaying the candidates.

  In the present embodiment, whether the search character string candidate is displayed by both the handwritten character string and the text, or by only the handwritten character string or only by the text, is determined by the user. Can be selected, and by performing a search process based on the handwritten character string or text selected by the user, it is possible to perform a search more reflecting the user's intention.

  Furthermore, in the present embodiment, a part of the search character string candidates is displayed on the screen according to the priority of each of the search character string candidates (that is, the points given for ranking). Accordingly, it is possible to suppress display of unnecessary search character string candidates, and it is possible to improve user convenience.

  Further, in the present embodiment, by adding (accumulating) points for each stroke (one stroke), the search character string with high accuracy (that is, intended by the user) each time the number of strokes (number of strokes) is overlapped. Candidates can be displayed.

  In the present embodiment, the description will be mainly given on the assumption that the search character string candidates are displayed on the screen when the user handwrites a character string (search character string) to be used as a search key for searching for a handwritten document. However, in the present embodiment, when a character string other than the search character string is handwritten, the character string candidates may be displayed. Furthermore, a configuration may be adopted in which candidates such as figures and tables are displayed in addition to the character string.

  Further, since the processing of this embodiment can be realized by a computer program, it is the same as that of this embodiment only by installing and executing this computer program on a computer through a computer-readable storage medium storing this computer program. The effect of can be easily realized.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 17 ... Touch screen display, 100 ... Pen, 101 ... CPU, 102 ... System controller, 103 ... Main memory, 105 ... BIOS-ROM, 106 ... Non-volatile memory, 201 ... OS, 202 ... Handwritten note application, 104 ... Graphics controller 107: Wireless communication device.

Claims (9)

And that the area for inputting a search key for searching the handwritten document, inputs the stroke data corresponding to a stroke set to be described by hand,
When n (n is the integer of 1 or more) first stroke data corresponding to the second stroke is input to the region, Rates stroke set from first included in the handwritten document is a search target through n Displaying on the screen a first candidate for the input character string;
When the second stroke data corresponding to n + 1-th stroke is input in the area, from the first included in the handwritten document is a search object is searched from the stroke set to n + 1 th, the character string to be input Displaying the second candidate on the screen,
Before Kite writing document, the time-series information indicating the order relationship between the coordinate series and the stroke of the locus of each stroke of the stroke set corresponding to the stroke data entered at the time the target stroke data is input Including
The first candidate and the second candidate character string to be the input from the stroke set and the first from the first on the basis of the time-series information included before Kite writing documents to n-th to (n + 1) th It is displayed on the screen as a search string candidate according to the stroke set,
After the second stroke data is input, if the first candidate for the input character string is different from the second candidate for the input character string, the input is performed more than the first candidate for the input character string. A method that preferentially displays the second candidate for the character string on the screen.   The method according to claim 1, wherein the second candidate for the input character string is displayed by at least one of a handwritten character string or text corresponding to the second candidate for the input character string.   Whether the second candidate for the input character string is displayed as the handwritten character string, the text, or both the handwritten character string and the text. The method of claim 2, wherein the method is selectable. An input unit for inputting stroke data corresponding to a stroke set that are described in the handwriting search area for inputting a key for searching the handwritten document,
When n (n is the integer of 1 or more) first stroke data corresponding to the second stroke is input to the region, Rates stroke set from first included in the handwritten document is a search target through n The first candidate for the input character string is displayed on the screen,
When the second stroke data corresponding to n + 1-th stroke is input in the area, from the first included in the handwritten document is a search object is searched from the stroke set to n + 1 th, the character string to be input A display control unit for displaying the second candidate on the screen,
Before Kite writing document, the time-series information indicating the order relationship between the coordinate series and the stroke of the locus of each stroke of the stroke set corresponding to the stroke data entered at the time the target stroke data is input Including
The first candidate and second candidate, stroke from the stroke set and the first from the first on the basis of the time-series information included before Kite writing documents to n-th to (n + 1) th string to the input It is displayed on the screen as a search string candidate according to the set,
After the second stroke data is input, if the first candidate for the input character string is different from the second candidate for the input character string, the input is performed more than the first candidate for the input character string. An electronic device that preferentially displays a second candidate for a character string on the screen. The electronic device according to claim 4 , wherein the second candidate character string to be input is displayed by at least one of a handwritten character string or text corresponding to the second candidate character string to be input. Whether the second candidate for the input character string is displayed as the handwritten character string, the text, or both the handwritten character string and the text. The electronic device according to claim 5 , which can be selected. A program executed by a computer, wherein the program
An input unit for inputting stroke data corresponding to a stroke set that are described by hand area for inputting a search key for searching the handwritten document,
When n (n is the integer of 1 or more) first stroke data corresponding to the second stroke is input to the region, Rates stroke set from first included in the handwritten document is a search target through n The first candidate for the input character string is displayed on the screen,
When the second stroke data corresponding to n + 1-th stroke is input in the area, from the first included in the handwritten document is a search object is searched from the stroke set to n + 1 th, the character string to be input Operate as a display controller that displays the second candidate on the screen,
The handwritten document includes time-series information indicating a coordinate sequence of a trajectory of each stroke of a stroke set corresponding to the stroke data input at the time when target stroke data is input and an order relationship between the strokes,
The first candidate and the second candidate character string to be the input from the stroke set and the first from the first on the basis of the time-series information included before Kite writing documents to n-th to (n + 1) th It is displayed on the screen as a search string candidate according to the stroke set,
After the second stroke data is input, if the first candidate for the input character string is different from the second candidate for the input character string, the input is performed more than the first candidate for the input character string. A program that preferentially displays the second candidate for a character string on the screen. The program according to claim 7 , wherein the second candidate character string to be input is displayed by at least one of a handwritten character string or text corresponding to the second candidate character string to be input. Whether the second candidate for the input character string is displayed as the handwritten character string, the text, or both the handwritten character string and the text. The program according to claim 8 , wherein the program is selectable.

Images