WO2020099297A1

WO2020099297A1 - Handwriting trajectory drawing method and device

Info

Publication number: WO2020099297A1
Application number: PCT/EP2019/080816
Authority: WO
Inventors: Yu Gao
Original assignee: Continental Automotive Gmbh
Priority date: 2018-11-12
Filing date: 2019-11-11
Publication date: 2020-05-22
Also published as: CN109558059A; CN109558059B

Abstract

A handwriting trajectory drawing method and device are provided. The handwriting trajectory drawing method comprises: a central processing unit receiving a plurality of touch events in response to handwriting input, the central processing unit generating a trajectory point sequence according to the plurality of touch events, the central processing unit sending a drawing command and the trajectory point sequence to a graphics processing unit, and the graphics processing unit drawing the handwriting trajectory according to the trajectory point sequence in response to the drawing command. By using the handwriting trajectory drawing method and device provided by the invention, a handwriting trajectory can be drawn quickly, and thus the user experience is improved.

Description

Handwriting Trajectory Drawing Method and Device

Technical Field

The invention relates to the field of man-machine interactions, and in particular relates to a handwriting trajectory drawing method and device.

Background Art

Currently, more and more devices, for example, smartphones and onboard infotainment systems, are adopting the touch mode for man-machine interactions. The devices adopting the touch mode for man-machine interactions usually have a touch board suitable for user input and a display displaying user input. When a user performs handwriting input on the touch board, the display will display the handwriting trajectory of the user so as to feed back the input of the user. If the time interval from when the user starts handwriting input to when the handwriting trajectory is displayed is large, the user will feel that the handwriting is not smooth and the user experience will be bad. In order to improve the user experience, it is necessary to shorten as much as possible the time interval from when the user starts handwriting input on the touch board to when the display displays the handwriting trajectory.

Summary of the Invention

The problem solved by the invention is to provide a handwriting trajectory drawing method and device, which can draw a handwriting trajectory quickly, thus improving the user experience.

To solve the above-mentioned problem, one aspect of the invention provides a handwriting trajectory drawing method suitable for a handwriting trajectory drawing device. The handwriting trajectory drawing device comprises a central processing unit and a graphics processing unit, and the method comprises: the central processing unit receiving a plurality of touch events in response to handwriting input, the central processing unit generating a trajectory point sequence according to the plurality of touch events, the central processing unit sending a drawing command and the trajectory point sequence to the graphics processing unit, and the graphics processing unit drawing the handwriting trajectory according to the trajectory point sequence in response to the drawing command.

Another aspect of the invention provides a handwriting trajectory drawing device. The handwriting trajectory drawing device comprises: a computer-readable storage medium, in which a plurality of commands are stored, one or more central processing units, suitable for executing the plurality of commands to perform the following actions: receiving a plurality of touch events in response to handwriting input, generating a trajectory point sequence according to the plurality of touch events, and sending a drawing command and the trajectory point sequence to one or more graphics processing units, suitable for drawing the handwriting trajectory according to the trajectory point sequence in response to the drawing command.

A further aspect of the invention provides a nonvolatile computer-readable storage medium, in which a plurality of commands suitable for one or more central processing units to execute are stored, the plurality of commands being executed by the one or more central processing units so that the one or more central processing units perform the following actions: receiving a plurality of touch events in response to handwriting input, generating a trajectory point sequence according to the plurality of touch events, and sending a drawing command and the trajectory point sequence to one or more graphics processing units.

Compared with the prior art, the above-mentioned solution has the following advantages:

For the handwriting trajectory drawing method and device provided by the invention, in one aspect, the position information of touch points is directly added to a trajectory point sequence, without any necessity of determining whether the distance between two adjacent touch points is greater than a preset value, and in another aspect, the related drawing function needs to be invoked only once in one drawing, without any necessity of invoking the drawing function repeatedly. These two aspects save the processing overhead of the central processing unit and speed up the processing so that a handwriting trajectory can be drawn quickly and the user experience can be improved.

Brief Description of the Drawings Fig. 1 exemplifies the modules of the handwriting trajectory drawing device according to one or more embodiments of the invention.

Fig. 2 exemplifies the time sequence of the handwriting trajectory drawing method according to one or more embodiments of the invention.

Fig. 3 exemplifies handwriting trajectories according to one or more

embodiments of the invention.

Detailed Description of the Invention

Details are given in the following description so that those skilled in the art can understand the invention more comprehensively. Flowever, it is obvious to those skilled in the art that the invention can be realized without some of these details. In addition, it should be understood that the invention is not limited to the specific embodiments described. On the contrary, the invention can be implemented by considering using any combination of the following characteristics and elements, no matter whether they relate to different embodiments. Therefore, the following aspects, characteristics, embodiments and advantages are only used to describe the invention, and should not be considered as the elements or limitations of the claims, unless otherwise expressly specified in the claims.

The inventor of the invention finds that if the mode in which the central processing unit repeatedly invokes the function for drawing line segments to let the graphics processing unit draw line segments repeatedly to draw a handwriting trajectory is adopted, a large time interval (for example, 250 ms) is required from when a user starts handwriting input to when the handwriting trajectory is displayed. After research, the inventor finds that the major reasons for a large time interval required for the foregoing handwriting trajectory drawing mode include: (1 ) the large time overhead required for each interaction of the central processing unit with the graphics processing unit and (2) the time overhead brought about by the operation of determining whether to discard some touch points according to whether the distance between two adjacent touch points is greater than a preset value (for example, 100 pixels). On the basis of the above-mentioned findings, the inventor proposed a handwriting trajectory drawing method and device which can be used to draw a handwriting trajectory quickly. Fig. 1 exemplifies the modules of the handwriting trajectory drawing device according to one or more embodiments of the invention. As shown in Fig. 1 , the handwriting trajectory drawing device 100 comprises a processing system 1 10 and a computer-readable medium 120. A plurality of commands and/or data are stored in the computer-readable medium 120. The processing system 1 10 may execute the plurality of commands stored in the computer-readable medium 120 so as to perform one or more actions, thus realizing various functions. For example, the processing system 1 10 may execute a plurality of commands so as to process one or more touch events, thus realizing the drawing of a handwriting trajectory. The processing system 1 10 may further read data from the computer-readable medium 120 and/or store data in the computer-readable medium 120. The processing system 1 10 may comprise a central processing unit (CPU) 1 1 1 and/or a graphics processing unit (GPU) 1 12. The CPU 1 1 1 is used to process general-purpose computing requirements. The GPU 1 12 is used to process graphic processing requirements. The computer-readable medium 120 may comprise a memory 121 and/or a storage 122. Examples of the memory 121 include a volatile storage medium, for example, a random access memory (RAM). Examples of the storage 122 include a nonvolatile storage medium, for example, a read only memory (ROM), a flash, a compact disc (CD) and a magnetic disk.

The handwriting trajectory drawing device 100 may further comprise a touch module 130. The touch module 130 is used to generate one or more touch events according to the operations of a user on the touch module. According to different operations of the user, touch events can be classified into different types, for example, touch start, touch move and touch end. A touch event contains position information of one or more touch points on the touch module 130 when the touch event is generated. The position information may include the coordinates based on a Cartesian coordinate system or the coordinates based on a polar coordinate system, for example. The operations of the user on the touch module 130 can be performed by a finger or a handwriting pen. Examples of the touch module 130 may include a capacitive touch module, a resistive touch module and an undulatory (for example, acoustic wave and infrared) touch module.

The handwriting trajectory drawing device 100 may further comprise a timer 140. The timer 140 is used for timing according to a timing request and is used to trigger an interruption when the timer expires.

The handwriting trajectory drawing device 100 may further comprise a display module 150. The display module 150 is used to display various interactive interfaces. When the user performs handwriting input, the display module 150 is further used to display the handwriting trajectory of the user.

The handwriting trajectory drawing device 100 may further comprise an input/output (I/O) interface 160. The I/O interface 160 is used to input and/or output signals, data or information. For example, the handwriting trajectory drawing device 100 may perform wired or wireless data interactions with an external device through the I/O interface 160. In one or more embodiments, the handwriting trajectory drawing device 100 is internally equipped with no touch module 130 and receives the touch events generated by an external touch device through the I/O interface 160. In one or more embodiments, the handwriting trajectory drawing device 100 is internally equipped with no display module 150 and outputs the content to be displayed to an external display device for displaying through the I/O interface 160.

The handwriting trajectory drawing device 100 can be realized by adopting different configurations. For example, the handwriting trajectory drawing device 100 can be realized as one or more of a personal computer, a laptop computer, a tablet computer, a mobile phone, a portable music player, a wearable smart device and an onboard infotainment system.

Fig. 2 exemplifies the time sequence of the handwriting trajectory drawing method according to one or more embodiments of the invention. The handwriting trajectory display method 300 can be realized in the handwriting trajectory drawing device 100 shown in Fig. 1 , for example. For example, the handwriting trajectory display method 300 is stored in the form of commands (for example, application) in the computer-readable medium 120 and is invoked and/or realized by the processing system 1 10. The steps of the handwriting trajectory display method 300 are described below in combination with Fig. 2.

In step S310, the touch module 130 generates one or more touch events in response to the handwriting input of the user. A touch event may contain the type of the touch event and the position information of the touch points corresponding to the touch event. The position information is the coordinates (x, y) in a Cartesian coordinate system, for example. Generally speaking, when a finger or handwriting pen of the user starts to touch the touch module 130, a touch event whose type is touch start will be generated. When the finger or handwriting pen moves on the touch module 130, one or more touch events whose type is touch move will be generated. When the finger or handwriting pen of the user leaves the touch module 130, a touch event whose type is touch end will be generated. It can be understood that touch events will also be continuously generated as touch operations continue.

In step S320, after receiving touch events, the CPU 1 1 1 generates a trajectory point sequence according to the touch events. To be specific, after receiving touch events, the CPU 1 1 1 extracts the position information of the touch point related to handwriting input from each touch event and adds the position information of each touch point to the trajectory point sequence. That is to say, in step S320, the position information of the touch points in all touch events is all added to the trajectory point sequence, without any necessity of determining whether the distance between two adjacent touch points is greater than a preset value. In one or more embodiments, the CPU 1 1 1 also adds the type of a touch event to the trajectory point sequence. In one or more embodiments, the trajectory point sequence can be stored in a computer-readable medium 120, for example, a memory 121 .

In step S330, the CPU 1 1 1 sends a drawing command (DrawCmd) and the trajectory point sequence to the GPU 1 12. In one or more embodiments, step S330 comprises step S332 of determining the drawing command to be sent according to the number of trajectory points in the trajectory point sequence. To be specific, in step S332, it is determined whether the trajectory point sequence contains only one trajectory point; if the trajectory point sequence contains only one trajectory point, the CPU 1 1 1 sends the drawing command for drawing a dot to the GPU 1 12; if the trajectory point sequence contains a plurality of trajectory points, the CPU 1 1 1 sends a drawing command for drawing a polyline to the GPU 1 12. In one or more embodiments, when the trajectory point sequence contains a plurality of trajectory points, the CPU 1 1 1 sends the drawing command for drawing a curve to the GPU 1 12. The drawing command for drawing a curve may include a command for drawing a Bezier curve, for example.

In one or more embodiments, the GPU 1 12 is a GPU supporting OpenVG. OpenVG is an application programming interface (API) designed for hardware accelerated vector graphics and is administrated by Khronos Group, a non-profit technical association. In such embodiments, the drawing command sent by the CPU 1 1 1 to the GPU 1 12 is a function invocation command of OpenVG. The drawing command for drawing a dot can be sent by invoking a function for drawing an ellipse. The invocation interface of the function for drawing an ellipse is as follows: vguErrorCode vguEllipse(VGPath path, VGfloat cx, VGfloat cy, VGfloat width, VGfloat height).

In one or more embodiments, the CPU 1 1 1 may send the drawing command for drawing a polyline to the GPU 1 12 by invoking the polygon drawing function having non-closed parameters. For the embodiment of GPU 1 12 supporting OpenVG, the invocation interface of the function for drawing a polygon is as follows: vguErrorCode vguPolygon(VGPath path, const VGfloat ^* points, VGint count, VGboolean closed).

In one or more embodiments, step S330 comprises step S331 of taking the position information of all touch points out of the trajectory point sequence. After the position information of all touch points is taken out of the trajectory point sequence, the trajectory point sequence is empty. In this way, while a trajectory is drawn based on the taken-out position information of all touch points, newly received touch events are simultaneously processed and the position information of the touch points in the newly received touch events is added to the trajectory point sequence. That is to say, the CPU 1 1 1 may simultaneously perform step S320 and step S330. It should be noted that after taking the position information of all trajectory points out of the trajectory point sequence, the CPU 1 1 1 sends a drawing command together with the taken-out position information of the trajectory points to the GPU 1 12.

In one or more embodiments, step S330 can be performed when an interruption is triggered. To be specific, when the preset length of the timer 140 expires, an interruption is triggered to cause the CPU 1 1 1 to perform step S330. In one or more embodiments, the timer 140 triggers an interruption at every preset length. The preset length can be 40 ms, for example.

In step S340, the GPU 1 12 draws the handwriting trajectory in response to the drawing command and the trajectory point sequence sent by the CPU 1 1 1 . To be specific, when the CPU 1 1 1 sends the drawing command for drawing a dot, the GPU 1 12 draws a dot according to the received position information of one touch point (step S341 ). When the CPU 1 1 1 sends the drawing command for drawing a polyline, the GPU 1 12 draws a polyline according to the received position information of a plurality of touch points (step S342), that is to say, joins every two adjacent touch points with a line segment to form a polyline. In one or more embodiments, when the CPU 1 1 1 sends the drawing command for drawing a curve to the GPU 1 12, the GPU 1 12 draws a curve (not shown in the figure) according to the received position information of a plurality of touch points. Since the GPU 1 12 includes the hardware accelerator for drawing various graphics, the GPU can quickly draw graphics such as a dot, a polyline and a curve.

In step S350, the CPU 1 1 1 sends the drawn handwriting trajectory to the display module 150 for display.

It should be noted that the above-mentioned steps S310 to S350 are described as a part of the drawing of the handwriting trajectory. Generally speaking, steps S310 to S350 will be repeatedly invoked many times during the drawing of the whole handwriting trajectory. As shown in Fig. 2, the touch module 130 generates touch events numbered from 0 to n+m during the touches of the user; the touch events numbered from 0 to n are processed and the handwriting trajectory corresponding to touch events numbered from 0 to n is drawn when steps S310 to S350 are invoked for the first time; the touch events number from n+1 to n+m are processed and the handwriting trajectory corresponding to touch events numbered from n+1 to n+m is drawn when steps S310 to S350 are invoked for the second time.

Compared with the mode in which the CPU repeatedly invokes the function for drawing line segments to let the GPU draw line segments repeatedly to draw a handwriting trajectory, the handwriting trajectory display method 300 saves the processing overhead of the CPU 1 1 1 at least in the following two aspects.

In one aspect, for the handwriting trajectory display method 300, the position information of touch points is directly added to a trajectory point sequence, without any necessity of determining whether the distance between two adjacent touch points is greater than a preset value. This saves the processing overhead of the CPU 1 1 1 and speeds up the processing.

In another aspect, the related drawing function needs to be invoked only once in one drawing (step S330) of the handwriting trajectory display method 300, without any necessity of invoking the drawing function repeatedly. This saves the processing overhead for cyclically invoking the function in, that is to say, the time for cyclic runs is saved and the processing speed is also improved. For example, for a trajectory point sequence containing n+1 touch points, if the mode in which the CPU repeatedly invokes the function for drawing line segments to let the GPU repeatedly draw line segments to draw the handwriting trajectory is adopted, n loops are required and the function for drawing line segments needs to be invoked n times; if the handwriting trajectory display method 300 is adopted, the function for drawing polylines needs to be invoked only once. Since the GPU 1 12 contains a hardware accelerator for drawing various graphics and the time for drawing line segments once is almost the same as the time for drawing polylines once, a handwriting trajectory can be quickly drawn by use of the handwriting trajectory display method 300.

Experiments show that when the mode in which the CPU repeatedly invokes the function for drawing line segments to let the GPU repeatedly draw line segments to draw the handwriting trajectory is adopted in an onboard infotainment system, the time interval required from when a user starts handwriting input to when the handwriting trajectory is displayed is about 250 ms, while if the handwriting trajectory display method 300 is used to draw a handwriting trajectory in the same onboard infotainment system, the time interval required from when the user starts handwriting input to when the handwriting trajectory is displayed is only about 50 ms. It can be seen that the handwriting trajectory display method 300 is about five times as fast as the mode in which the CPU repeatedly invokes the function for drawing line segments to let the GPU repeatedly draw line segments to draw the handwriting trajectory.

Fig. 3 exemplifies handwriting trajectories according to one or more embodiments of the invention. Fig. 3(A) shows the handwriting trajectory drawn by use of the mode in which the CPU repeatedly invokes the function for drawing line segments to let the GPU repeatedly draw the handwriting trajectory, and it can be seen that the trajectory point P is displayed only 250 ms after the finger continues to move right. Fig. 3(B) shows the handwriting trajectory drawn by use of the handwriting trajectory display method 300, and it can be seen that the trajectory point P is displayed 50 ms after the finger continues to move right. It can be seen that when the handwriting trajectory display method 300 is adopted to draw a handwriting trajectory, the handwriting trajectory displayed on the display module follows the finger movement more closely, the handwriting of the user is smoother, and the user experience is improved.

Although the invention is already disclosed above with preferred embodiments, the invention is not limited to the preferred embodiments. Any change or modification made by those skilled in the art without departing from the spirit and scope of the invention should fall within the scope of protection of the invention. Therefore, the scope of protection of the invention should be subject to the scope defined by the claims.

Claims

Patent claims

1 . A handwriting trajectory drawing method, applicable to a handwriting trajectory drawing device, the handwriting trajectory drawing device comprising a central processing unit and a graphics processing unit, wherein the method comprises:

the central processing unit receiving a plurality of touch events in response to handwriting input,

the central processing unit generating a trajectory point sequence according to the plurality of touch events,

the central processing unit sending a drawing command and the trajectory point sequence to the graphics processing unit, and

the graphics processing unit drawing the handwriting trajectory according to the trajectory point sequence in response to the drawing command.

2. The handwriting trajectory drawing method as claimed in claim 1 , wherein the central processing unit sends the drawing command for drawing a polyline to the graphics processing unit.

3. The handwriting trajectory drawing method as claimed in claim 2, wherein the central processing unit sends the drawing command for drawing a polyline to the graphics processing unit by invoking a polygon drawing function, wherein the polygon drawing function has non-closed parameters.

4. The handwriting trajectory drawing method as claimed in claim 1 , wherein the central processing unit sends the drawing command for drawing a curve to the graphics processing unit.

5. The handwriting trajectory drawing method as claimed in claim 1 , wherein the step of the central processing unit sending a drawing command and the trajectory point sequence to the graphics processing unit is performed when an interruption is triggered.

6. The handwriting trajectory drawing method as claimed in claim 1 , wherein the step of the central processing unit generating a trajectory point sequence according to the plurality of touch events comprises:

extracting position information related to the handwriting input from each of the touch events, and adding the position information to the trajectory point sequence.

7. The handwriting trajectory drawing method as claimed in claim 1 or 6, wherein the touch events include one or more of touch start, touch move and touch end.

8. A handwriting trajectory drawing device, wherein the device comprises:

a computer-readable storage medium, in which a plurality of commands are stored,

one or more central processing units, suitable for executing the plurality of commands to perform the following actions:

receiving a plurality of touch events in response to handwriting input, generating a trajectory point sequence according to the plurality of touch events, and

sending a drawing command and the trajectory point sequence to one or more graphics processing units, and

the one or more graphics processing units, suitable for drawing the handwriting trajectory according to the trajectory point sequence in response to the drawing command.

9. A non-volatile computer-readable storage medium, in which a plurality of commands suitable for one or more central processing units to execute are stored, the plurality of commands being executed by the one or more central processing units so that the one or more central processing units perform the following actions:

receiving a plurality of touch events in response to handwriting input, generating a trajectory point sequence according the plurality of touch events, and

sending a drawing command and the trajectory point sequence to one or more graphics processing units.