JP7533534B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

A technology has been disclosed that uses a GPS (Global Positioning System) to detect location information, displays a moving object on a user's movement trajectory based on the location information and time information of a captured image, and displays the target image when the moving object reaches the display position of the captured image (see Patent Document 1).

JP 2015-69313 A

However, in conventional technology, a moving target is displayed on the user's movement trajectory based on the position information of the captured image taken by the user, and the target image is displayed when the moving target reaches the display position of the captured image. However, the target image is displayed in a uniform display format, resulting in a monotonous display.

The present invention was made in consideration of these circumstances, and aims to display a variety of additional information when playing back a movement trajectory.

In order to achieve the above object, an information processing device according to one aspect of the present invention comprises:
The device is equipped with a structured data generation means for generating structured data for displaying a trajectory of the movement history and content related to the movement history within a map display provided as map information based on data of activity history including a user's movement history and the type of the activity, wherein the structured data is characterized in that an event related to the display of the content is set at a predetermined position on the trajectory of the movement history, and an operation at the time of the event is set, including a time of change in the movement speed of a point of interest tracing the trajectory of the movement history at the predetermined position or whether or not to zoom in, based on the type of activity .

According to the present invention, it is possible to display a variety of additional information when playing back a movement trajectory.

1 is a schematic diagram showing a system configuration of an information processing system according to an embodiment of the present invention; FIG. 2 is a block diagram showing a hardware configuration of the information processing device. FIG. 2 is a block diagram showing a hardware configuration of the information processing device. 3 is a functional block diagram showing a functional configuration for executing a history recording process and a structured data generating process, among the functional configurations of a wrist terminal constituted by the information processing device of FIG. 2; FIG. FIG. 13 is a schematic diagram showing an example of a video attribute table. FIG. 13 is a schematic diagram showing an example of structured data for outputting position information. 4 is a functional block diagram showing a functional configuration for executing structured data analysis processing, among the functional configurations of a map information providing server constituted by the information processing device of FIG. 3; 4 is a functional block diagram showing a functional configuration for executing storage service processing, among the functional configurations of a cloud storage server constituted by the information processing device of FIG. 3. FIG. 4 is a functional block diagram showing a functional configuration for executing a video playback process, among the functional configurations of a PC constituted by the information processing device of FIG. 3. FIG. 4 is a schematic diagram showing an example of activity history data. 11 is a schematic diagram showing a display example in which the data of the activity history shown in FIG. 10 is converted into a video. FIG. FIG. 13 is a schematic diagram showing the operation of a photography event at position P4 where a photograph was taken. FIG. 13 is a schematic diagram showing a state in which an image of a photograph taken at position P4 is displayed in a pop-up. 5 is a flowchart illustrating the flow of a history recording process executed by a wrist terminal having the functional configuration of FIG. 4 . 5 is a flowchart illustrating the flow of a structured data generation process executed by a wrist terminal having the functional configuration of FIG. 4 . 8 is a flowchart illustrating the flow of a structured data analysis process executed by a map information providing server having the functional configuration of FIG. 7; 9 is a flowchart illustrating the flow of a storage service process executed by a cloud storage server having the functional configuration of FIG. 8 . 10 is a flowchart illustrating the flow of a video playback process executed by a PC having the functional configuration of FIG. 9 .

The following describes an embodiment of the present invention with reference to the drawings.

FIG. 1 is a schematic diagram showing a system configuration of an information processing system 1 according to an embodiment of the present invention.
As shown in FIG. 1 , the information processing system 1 includes a wrist terminal 10 , an image capturing device 20 , a smartphone 30 , a map information providing server 40 , a cloud storage server 50 , and a PC (Personal Computer) 60 .
Among these, the wrist terminal 10, the photographing device 20, and the smartphone 30 are configured to be able to communicate with each other by wireless communication such as Bluetooth low energy/Bluetooth LE (registered trademark) (hereinafter referred to as "BLE") or Wi-Fi (Wireless Fidelity). In addition, the smartphone 30, the map information providing server 40, the cloud storage server 50, and the PC 60 are configured to be able to communicate with each other via a network 70 such as the Internet. In this embodiment, the wrist terminal 10, the photographing device 20, and the smartphone 30 may also have a function of communicating with other devices via the network 70 such as the Internet.

Below, an overview of each device constituting the information processing system 1 will be described.
The wrist terminal 10 is configured as a wristwatch-type information processing device (so-called a smart watch) and is worn on the arm of a user who is engaged in activities such as mountain climbing or road biking (cycling). The wrist terminal 10 also has a GPS positioning function and executes a process of recording a history of an activity including a user's movement history (hereinafter referred to as a "history recording process"), and a process of generating structured data (hereinafter referred to as "structured data for outputting position information") that is described to display position information and content related to the position information based on the recorded history data (hereinafter referred to as a "structured data generation process"). For example, when a user wearing the wrist terminal 10 performs various activities, the wrist terminal 10 executes the history recording process to record data of the activity history including the movement history in association with the type of activity. The wrist terminal 10 also executes the structured data generation process to automatically generate structured data for outputting position information that is described to display position information and content related to the position information based on the recorded history data. Then, the wrist terminal 10 uploads the generated structured data for outputting position information to the cloud storage server 50 via the smartphone 30 .

The photographing device 20 is configured as an information processing device equipped with a photographing function such as a digital camera. The photographing device 20 photographs an image (still image or video) according to a user's photographing instruction, and stores the photographed image data in association with the photographing date and time. In this embodiment, the photographing device 20 uploads the image data associated with the photographing date and time to the cloud storage server 50 via the smartphone 30 in accordance with a user's instruction or automatically according to the settings.

The smartphone 30 is configured as an information processing device equipped with a calling function and an information processing function. The smartphone 30 uploads structured data for outputting location information transmitted from the wrist terminal 10, image data transmitted from the photographing device 20, etc., to the cloud storage server 50 via the network 70.

The map information providing server 40 is composed of an information processing device such as a server computer or a PC. The map information providing server 40 has a function of providing three-dimensional map information, and executes a process (hereinafter referred to as "structured data analysis process") of interpreting the structured data for outputting positional information and displaying map information and related content while moving the positions described in the structured data for outputting positional information in chronological order. In this embodiment, the structured data for outputting positional information describes the position of the point of interest (latitude, longitude, altitude), the moving speed of the viewpoint, the altitude of the viewpoint, the direction of the viewpoint (the direction of the line of sight), the horizontal angle of the viewpoint (depression angle), the actions during various events, and other video attributes. The map information providing server 40 outputs video data that traces the movement history while changing the position in the map information or performing a specified action according to the described video attributes. When performing the structured data analysis process, the map information providing server 40 obtains content such as captured images and text of spoken comments from the storage destination of the content described in the structured data for outputting location information (here, cloud storage server 50), and displays the obtained content at a specified position in the movement history.

The cloud storage server 50 is composed of an information processing device such as a server computer or a PC. The cloud storage server 50 is a server that provides cloud storage services, and executes a process (hereinafter referred to as "storage service process") of storing structured data for outputting location information and content (images, audio, text, etc.) uploaded by users in association with the user's identification information, and transmitting the data to other devices upon request.

The PC 60 is an information processing device used by a user, and executes a process for playing a video based on the structured data for outputting location information (hereinafter referred to as a "video playback process") in response to a user's instruction. For example, the PC 60 transmits a request to play a video together with the structured data for outputting location information to the map information providing server 40. The PC 60 also receives and displays video data that is the result of execution of the structured data analysis process in the map information providing server 40.

[Hardware configuration of information processing device]
Next, the hardware configuration of each device constituting the information processing system 1 will be described.
In the present embodiment, as an example, the wrist terminal 10, the photographing device 20, and the smartphone 30 constituting the information processing system 1 are configured by a portable information processing device 100, and the map information providing server 40, the cloud storage server 50, and the PC 60 are configured by a stationary information processing device 200. Hereinafter, the hardware configurations of the information processing device 100 constituting the wrist terminal 10, the photographing device 20, and the smartphone 30, and the information processing device 200 constituting the map information providing server 40, the cloud storage server 50, and the PC 60 will be described. Note that the hardware configurations of the information processing device 100 and the information processing device 200 can be appropriately changed depending on the device to be specifically implemented.

FIG. 2 is a block diagram showing the hardware configuration of the information processing device 100. As shown in FIG.
As shown in FIG. 2 , the information processing device 100 includes a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a bus 114, an input/output interface 115, a GPS receiving unit 116, an imaging unit 117, a sensor unit 118, an input unit 119, an output unit 120, a memory unit 121, a communication unit 122, and a drive 123.

The CPU 111 executes various processes according to a program recorded in the ROM 112 or a program loaded from the memory unit 121 to the RAM 113.

The RAM 113 also stores data and other information necessary for the CPU 111 to execute various processes.

The CPU 111, ROM 112, and RAM 113 are interconnected via a bus 114. An input/output interface 115 is also connected to this bus 114. A GPS receiver 116, an image capture unit 117, a sensor unit 118, an input unit 119, an output unit 120, a memory unit 121, a communication unit 122, and a drive 123 are connected to the input/output interface 115.

The GPS receiver 116 detects the position (latitude, longitude, altitude) of the information processing device 100 and the current time indicated by the GPS based on the GPS signals received from GPS satellites.

Although not shown, the imaging unit 117 includes an optical lens unit and an image sensor.

The optical lens section is composed of lenses that collect light in order to photograph a subject, such as a focus lens and a zoom lens.
A focus lens is a lens that focuses a subject image on the light receiving surface of an image sensor, while a zoom lens is a lens that can freely change the focal length within a certain range.
The imaging unit 117 is also provided with peripheral circuits for adjusting setting parameters such as focus, exposure, and white balance, as necessary.

The image sensor is composed of a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element is, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element. An object image is incident on the photoelectric conversion element from the optical lens unit. The photoelectric conversion element photoelectrically converts (captures) the object image, accumulates the image signal for a certain period of time, and sequentially supplies the accumulated image signal to the AFE as an analog signal.
The AFE performs various signal processing such as A/D (Analog/Digital) conversion processing on the analog image signal. A digital signal is generated by the various signal processing, and is appropriately supplied to the CPU 111 or an image processing unit (not shown) as an output signal of the imaging unit 117.

The sensor unit 118 includes various sensors such as an air pressure sensor, a temperature sensor, a humidity sensor, a wind speed sensor, a biosensor (a pulse sensor, a blood pressure sensor, etc.), a geomagnetic sensor, a three-axis acceleration sensor, and a three-axis angular velocity sensor. These various sensors included in the sensor unit 118 can be implemented either as built-in sensors in the information processing device 100 or as sensors connected to an external unit.

The input unit 119 is made up of various buttons and the like, and is used to input various pieces of information in response to user instructions.
The output unit 120 is composed of a display, a speaker, etc., and outputs images and sounds.
In this embodiment, the input unit 119 of the information processing device 100 includes a touch sensor, and the touch sensor is arranged so as to overlap the display of the output unit 120 to form a touch panel.
The storage unit 121 is configured with a semiconductor memory such as a dynamic random access memory (DRAM), and stores various types of data.
The communication unit 122 controls communication with other devices via a network including the Internet, and also controls communication with other devices by direct wireless communication between terminals.

Removable media 131, which may be a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately attached to drive 123. Programs read from removable media 131 by drive 123 are installed in storage unit 121 as necessary. In addition, removable media 131 can also store various data stored in storage unit 121 in the same way as storage unit 121.

FIG. 3 is a block diagram showing the hardware configuration of the information processing device 200. As shown in FIG.
As shown in FIG. 3, the information processing device 200 includes a CPU 211, a ROM 212, a RAM 213, a bus 214, an input/output interface 215, an input unit 216, an output unit 217, a memory unit 218, a communication unit 219, and a drive 220.
In the information processing device 200 shown in FIG. 3, the hardware configuration of each unit is similar to the corresponding hardware configuration of the information processing device 100 shown in FIG.
However, in this embodiment, the communication unit 219 may omit the function of communicating with other devices by direct wireless communication between terminals. Similarly to the drive 123, the drive 220 is appropriately equipped with a removable medium 231 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like.

[Functional configuration]
Next, the functional configuration of the information processing system 1 will be described.
FIG. 4 is a functional block diagram showing the functional configuration for executing the history recording process and the structured data generating process, among the functional configurations of the wrist terminal 10 constituted by the information processing device 100 of FIG.
The history recording process is a series of processes for recording the activity history including the user's movement history, and the structured data generation process is a series of processes for generating structured data for outputting location information, which is written to display location information and content related to the location information, based on the recorded history data.

When the history recording process and the structured data generation process are executed, as shown in FIG. 4, a user interface display control unit (UI display control unit) 51, a history acquisition unit 52, an activity determination unit 53, a video attribute setting unit 54, a parameter correction receiving unit 55, a structured data generation unit 56, and a structured data registration unit 57 function in the CPU 111.
Furthermore, a history data storage section 71 is set in one area of the storage section 121 .
The history data storage unit 71 stores data on the history of activities including the user's movement history.

The UI display control unit 51 displays a user interface screen for the user to input and output information on the wrist terminal 10. The UI display control unit 51 also accepts various operations via the user interface screen. For example, when the user performs an operation to start an activity, the UI display control unit 51 displays a list of multiple types of activity candidates and accepts the user's selection of an activity. When the user inputs an instruction to generate structured data for outputting location information from data of the activity history, the UI display control unit 51 also accepts input of content to be manually associated with the data of the activity history (such as input of an address of a storage area).
The history acquisition unit 52 sequentially acquires the position (latitude, longitude, altitude) and the current time detected by the GPS receiving unit 116, and stores them in the history data storage unit 71 as the user's movement history.

The activity determination unit 53 determines the type of activity performed by the user. In this embodiment, the activity determination unit 53 determines the type of activity performed by the user based on the activity type data included in the activity history data. Note that the activity determination unit 53 can also determine the type of activity performed by the user by detecting the type of activity selected by the user from a list of activity candidates via the user interface screen.

The video attribute setting unit 54 sets video attributes (position of the focus point (latitude, longitude, altitude), viewpoint movement speed, viewpoint altitude, viewpoint direction (direction of line of sight), viewpoint horizontal angle (depression angle), actions during various events, etc.) according to the type of activity performed by the user. In this embodiment, the type of activity and the video attributes corresponding to that type of activity are set in advance as table-format data (hereinafter referred to as a "video attribute table"). The video attribute setting unit 54 then sets the video attributes by reading the video attributes corresponding to the type of activity from the video attribute table.

FIG. 5 is a schematic diagram showing an example of a video attribute table.
As shown in FIG. 5, the video attribute table stores types of activities and attributes of videos corresponding to the types of activities in association with each other.
In this embodiment, mountain climbing, road biking (cycling), skiing/snowboarding, paddle boarding, surfing, and off-road biking are defined as activity types.

As a general example, when climbing mountains, the speed is 1-3 km/h, the altitude is 0-8000 m, and the distance traveled is about 15 km. When riding a road bike, the speed is 10-60 km/h, and the distance traveled is long (about 20 km or more). When skiing/snowboarding, the speed is 20-50 km/h, and the distance traveled in one go is short (about 10 km or less). When paddleboarding, the speed is 2-10 km/h, and the distance traveled is indefinite. When surfing, the speed is 10-40 km/h, and the distance traveled in one go is about 100 m or less. When off-road biking, the speed is 20-100 km/h, and the distance traveled is indefinite.

In consideration of these characteristics, in this embodiment, video attributes include the position of the focus point (latitude, longitude, altitude), viewpoint movement speed, viewpoint altitude, viewpoint direction (direction the line of sight is facing), viewpoint horizontal angle (depression angle), and actions during various events (for example, whether to pause at a specific point, zooming in or diving when paused, or stopping time when paused), and values and actions are defined according to the type of activity.

For example, regarding the position of the point of interest, each position in the movement history of the activity or a selected specific position can be defined as a target position for shooting a virtual camera in a video.
Furthermore, the speed of movement of the viewpoint can be defined as a constant multiple of the actual movement speed in the activity or a specific selected speed, or it can be defined as the speed at which the entire process is moved in a preset time (e.g., 3 minutes).
Regarding the altitude of the viewpoint, in the case of mountain climbing or trekking, a viewpoint height that allows a wide view of the range of movement can be defined, and in the case of road biking (cycling), a viewpoint height that allows the sense of speed of the bicycle can be defined (i.e., a viewpoint height that is lower than that of mountain climbing or trekking).

As for the viewpoint direction, the direction in which the line of sight (the shooting direction of a virtual camera in a video) faces when displaying a point of interest can be defined according to the type of activity.
Regarding the horizontal angle of view, a depression angle can be defined according to the type of activity. For example, for mountain climbing or trekking, a relatively large depression angle (an angle similar to looking straight down) can be defined, while for road biking (cycling), a relatively small depression angle (an angle similar to looking in the direction of travel) can be defined.

Regarding the actions during various events, for example, the action at the point where a photo was taken, in the case of mountain climbing or trekking, is to temporarily stop moving for a relatively long time (e.g., 3 seconds, etc.), temporarily zoom in on the shooting location, and then display the image of the taken photo in a pop-up, or in the case of road biking, is to temporarily stop moving for a relatively short time (e.g., 1.5 seconds), and display the image of the taken photo in a pop-up without zooming in on the shooting location. Also, for example, the action at the point where the user inputs an utterance (inputs voice into the wrist terminal 10), in the case of mountain climbing or trekking, is to slow down the moving speed for a relatively long time (e.g., 3 seconds, etc.), and display the text data of the uttered voice in a speech bubble (balloon), or in the case of road biking, is to slow down the moving speed for a relatively short time (e.g., 1.5 seconds), and display the text data of the uttered voice in a speech bubble (balloon). In addition, for the start and finish points, or comments that the user manually adds to specific points in the history data after completing an activity, a text box containing a marker and a comment about that point can be displayed in a pop-up, and the display format can be varied depending on the type of activity.

The parameter correction receiving unit 55 receives a user's correction operation for the parameters representing the attributes of the video set by the video attribute setting unit 54. For example, the parameter correction receiving unit 55 changes parameters, such as the movement speed, viewpoint altitude, or horizontal angle of the viewpoint, set by the video attribute setting unit 54, in response to a user's operation.

When an instruction to generate structured data for outputting location information for data of a user's activity history is input, the structured data generation unit 56 generates structured data for outputting location information for displaying data of the movement history (location information) in the activity and content related to the movement history, based on the video attributes set in the data of the activity history. In this embodiment, the content related to the movement history is content that the user has manually associated by inputting the address of a storage area, etc.

FIG. 6 is a schematic diagram showing an example of structured data for outputting position information.
As shown in Fig. 6, the structured data for outputting position information is configured as a structured document whose structure is defined by tags, and various tags are defined assuming the output of position information. Such a structured document can be written in, for example, XML (extensible Markup Language) format, and various tags effective for outputting position information, such as a tag for setting a viewpoint and a tag for defining an action at the time of an event, can be defined.
In this embodiment, the structured data for outputting position information is composed of a plurality of blocks grouped by function, as shown in Fig. 6. As an example, the structured data for outputting position information includes a block for defining various styles, a block for defining the drawing of trajectory lines, a block for defining the drawing of markers, a block for defining the display of videos, and the like.

The blocks that define various styles define the style of various elements in the display of the video, such as the URL where the image of a marker as an element in the display of the video is stored, the size of the marker, or the transparency of the marker.
The block that defines the drawing of the trajectory line defines the drawing of the trajectory displayed in the video, and defines multiple coordinates that represent the user's movement history consecutively in the chronological order of the movement. In addition, the block that defines the drawing of the trajectory line defines the line width and line color of the trajectory.

The block that defines the drawing of markers defines the drawing of markers to be displayed on the trajectory, and defines the markers to be added to the trajectory, text data of spoken voice, information about the photo, information about the start position and goal position, etc., the display position on the movement trajectory and the corresponding time (such as the time the photo was taken), the style to be applied, the URL where the actual content to be drawn (such as a photo) is stored, etc.

The block that defines the video display defines the video's presentation, and defines the operation of displaying or erasing markers in the video (operations including various presentations), the position of the point of interest (latitude, longitude, altitude), viewpoint movement speed, viewpoint altitude, horizontal angle of the viewpoint, and altitude reference (altitude above sea level, etc.). When displaying the video, the view (map information) of the position of the point of interest as viewed from the defined viewpoint altitude, horizontal angle, etc. is displayed while moving sequentially to each defined viewpoint position at the defined viewpoint speed.

The structured data registration unit 57 uploads the structured data for outputting location information generated by the structured data generation unit 56 to the cloud storage server 50 via the smartphone 30.

FIG. 7 is a functional block diagram showing a functional configuration for executing structured data analysis processing, among the functional configurations of the map information providing server 40 constituted by the information processing device 200 of FIG.
The structured data analysis process is a series of processes that interpret structured data for outputting location information and display map information and related content, etc., while moving the viewpoint position described in the structured data for outputting location information in chronological order.

When the structured data analysis process is executed, as shown in FIG. 7, the structured data receiving unit 151, the structured data executing unit 152, and the content acquisition unit 153 function in the CPU 211.

The structured data receiving unit 151 receives structured data for outputting position information to be processed from the PC 60 .
The structured data execution unit 152 interprets the structured data for outputting location information received by the structured data reception unit 151, and specifies the address of the related content (such as the URL (Uniform Resource Locator) of the storage area in the cloud storage server 50). The structured data execution unit 152 acquires the related content from the specified address via the content acquisition unit 153. The structured data execution unit 152 then executes the structured data for outputting location information, and generates video data for displaying map information and related content, etc., while moving the position of the viewpoint described in the structured data for outputting location information in chronological order. At this time, since the structured data for outputting location information describes the attributes of the video according to the type of activity, the generated video data has a display form appropriate to the activity. The structured data execution unit 152 then transmits the generated video data to the PC 60 that transmitted the structured data for outputting location information.
The content acquisition unit 153 acquires various contents from the addresses of the related contents identified by the structured data execution unit 152 by interpreting the structured data for outputting position information.

FIG. 8 is a functional block diagram showing a functional configuration for executing storage service processing, among the functional configurations of the cloud storage server 50 constituted by the information processing device 200 of FIG.
The storage service processing is a series of processes for storing various data uploaded by the user, such as structured data for outputting location information sent from the wrist terminal 10 and image data sent from the photographing device 20, in association with the user, and for providing the various stored data in response to requests from other devices, such as the map information providing server 40.

When storage service processing is executed, as shown in FIG. 8, a structured data receiving unit 251, a structured data transmitting unit 252, a content receiving unit 253, a content transmitting unit 254, and a data management unit 255 function in the CPU 211.
In addition, a structured data storage section 271 and a related content storage section 272 are set in one area of the storage section 218 .
The structured data storage unit 271 stores structured data for outputting position information uploaded by a user in association with the user's identification information.
The related content storage unit 272 stores data of various contents uploaded by users in association with the user's identification information.

The structured data receiving unit 251 receives structured data for outputting location information transmitted from another device such as the smartphone 30, and outputs the received structured data for outputting location information to the data management unit 255 together with the identification information of the user who transmitted it.
When the structured data transmission unit 252 receives a transmission request for structured data for outputting location information from another device such as the smartphone 30, the structured data transmission unit 252 outputs the received transmission request to the data management unit 255. The transmission request for structured data includes the user's identification information and the identification information of the structured data requested to be transmitted. Then, when the structured data transmission unit 252 receives structured data for outputting location information corresponding to the transmission request from the data management unit 255, the structured data transmission unit 252 transmits the input structured data for outputting location information to the device that requested the transmission.

The content receiving unit 253 receives content transmitted from another device such as the smartphone 30, and outputs the received content to the data management unit 255 together with the identification information of the user who transmitted the content.
When the content transmission unit 254 receives a content transmission request from another device such as the PC 60, it outputs the received transmission request to the data management unit 255. The content transmission request includes the user's identification information and the identification information of the content requested to be transmitted. When the structured data transmission unit 252 receives the content corresponding to the transmission request from the data management unit 255, it transmits the input content to the device that has sent the transmission request.

When the data management unit 255 receives structured data for outputting location information and the identification information of the user who sent it from the structured data receiving unit 251, the data management unit 255 associates the structured data for outputting location information with the user's identification information and stores them in the structured data storage unit 271. Furthermore, when the data management unit 255 receives a request to send structured data for outputting location information from the structured data sending unit 252, it reads out, from the structured data storage unit 271, the structured data for outputting location information that is associated with the user's identification information indicated in the request to send, and outputs it to the structured data sending unit 252.

When the data management unit 255 receives content and the identification information of the user who sent it from the content receiving unit 253, the data management unit 255 associates the content with the user's identification information and stores them in the related content storage unit 272. When the data management unit 255 receives a content transmission request from the content transmission unit 254, the data management unit 255 reads out the content associated with the user's identification information indicated in the transmission request from the related content storage unit 272, and outputs it to the content transmission unit 254.

FIG. 9 is a functional block diagram showing a functional configuration for executing a video playback process, among the functional configurations of the PC 60 constituted by the information processing device 200 of FIG.
The video playback process is a series of processes for playing back a video based on structured data for outputting position information in response to a user instruction.

When the video playback process is executed, as shown in FIG. 9, a structured data acquisition unit 351, a video playback request unit 352, and a video playback unit 353 function in the CPU 211.
In addition, a structured data storage section 371 is set in one area of the storage section 218 .
The structured data storage unit 371 stores structured data for outputting position information that is to be the subject of video playback.

The structured data acquisition unit 351 transmits a request to the cloud storage server 50 to transmit structured data for outputting location information in response to a video playback instruction issued by the user. The structured data acquisition unit 351 then receives structured data for outputting location information corresponding to the transmission request from the cloud storage server 50, and stores the received structured data for outputting location information in the structured data storage unit 371.

The video playback request unit 352 transmits a request to play the video to the map information providing server 40 together with the structured data for outputting position information for which playback of the video has been instructed by the user.
The video playback unit 353 receives video data based on the structured data for position information output transmitted from the map information providing server 40 , and plays the data on the display of the output unit 217 .

[Specific display examples]
FIG. 10 is a schematic diagram showing an example of activity history data.
In the example shown in Fig. 10, the entire data of the history of a case where mountain climbing was performed as an activity is shown, and various markers are added to positions P1 to P6 on the trajectory of the movement history. Note that the thick dashed lines in Fig. 10 represent the trajectory in the movement history (similarly in Figs. 11 to 13).
For example, the marker at position P1 is a marker representing a start position, and displays the text "Start of mountain climbing 10:40:00". The marker at position P2 is a marker representing the text of a comment added by a user, and displays the text "Comment 10:46:45". The marker at position P3 is a marker representing the text of a comment input by a user through speech recognition, and displays the text "Speech 12:36:54". The marker at position P4 is a marker representing a position where a photo was taken, and displays the text "Photo 13:04:22". The marker at position P5 is a marker representing a position where a photo was taken, and displays the text "Photo 13:48:04". The marker at position P6 is a marker representing a goal position, and displays the text "Goal of mountain climbing 14:28:47".

Such activity history data is converted by the wrist terminal 10 into structured data for outputting location information, in which video attributes according to the type of activity are set, and is interpreted and executed by the map information providing server 40 to produce a video in an appropriate display format according to the type of activity.

FIG. 11 is a schematic diagram showing a display example in which the data of the activity history shown in FIG. 10 is displayed as a video.
In Fig. 11, a dotted circle indicates the position of the attention point. Note that the circle indicating the attention point is shown to show the position of the attention point in a schematic manner, but is not displayed in the actual video in this embodiment. However, the position of the attention point may be explicitly shown by displaying a moving icon or the like.

As shown in Figure 11, when the data of the mountain climbing history shown in Figure 10 is made into a video, the position of the focus point is moved sequentially from the starting position, and a viewpoint set at a high altitude that overlooks a relatively wide range is displayed in a manner that traces the movement history.
When the attention point reaches a position on the trajectory of the movement history where an event is set, the action of the set event is executed.

FIG. 12 is a schematic diagram showing the operation of a photography event at position P4 where the photograph was taken.
12, when the attention point reaches the position P4 where the photograph was taken, the movement of the viewpoint and attention point is temporarily stopped for a relatively long time (e.g., 3 seconds, etc.) Then, after the photographing point (position P4) is temporarily zoomed in, the image of the photograph is displayed in a pop-up.

FIG. 13 is a schematic diagram showing a state in which an image of a photograph taken at position P4 is displayed in a pop-up.
In the example shown in FIG. 13, text indicating a comment entered by the user is displayed together with the captured photograph.

As shown in FIG. 13, when the point of interest reaches position P4 where the photograph was taken, movement of the viewpoint and point of interest is paused for a relatively long time (e.g., 3 seconds), and then the image is zoomed in on the photographed location and a magnified image of the area around the photographed location is displayed. Then, a pop-up containing the photograph associated with position P4 is displayed for a predetermined time. After that, the pop-up display is cleared, and the image returns from the zoomed-in state to the original viewpoint position, and the video display continues.

In the example shown in FIG. 13, the pop-up containing the photo is displayed so as not to overlap with the movement history trajectory. This makes it easy to understand which position in the movement history trajectory the photo corresponds to. Note that the photo associated with position P4 may be displayed in a larger size or across the entire screen. In this case, the photo associated with position P4 can be displayed in a form that is easier to view.

[Action]
Next, the operation of the information processing system 1 will be described.
[History Recording Process]
FIG. 14 is a flowchart illustrating the flow of a history recording process executed by the wrist terminal 10 having the functional configuration of FIG.
The history recording process is started when an operation to instruct the start of the history recording process is performed via the input unit 119 .

In step S1, the UI display control unit 51 displays a user interface screen for the user to input and output information.
In step S2, the UI display control unit 51 receives an input of the type of activity to be the target of the history recording process. At this time, the UI display control unit 51 displays a list of multiple types of activity candidates and receives the user's selection of an activity.

In step S3, the history acquisition unit 52 acquires the position (latitude, longitude, altitude) and the current time detected by the GPS receiving unit 116.
In step S4, the history acquisition unit 52 determines whether or not an instruction to end the history recording process has been issued.
If the end of the history recording process has not been instructed, the result of the determination in step S4 is NO, and the process proceeds to step S3.
On the other hand, if an instruction to end the history recording process is given, a YES determination is made in step S4, and the process proceeds to step S5.

In step S5, the history acquisition unit 52 associates the acquired series of position data with the acquisition time, and stores the data as the user's movement history in the history data storage unit 71. Note that the user's movement history stored at this time includes the data on the type of activity accepted in step S2.
After step S5, the history recording process ends.

[Structured data generation process]
FIG. 15 is a flowchart illustrating the flow of a structured data generation process executed by the wrist terminal 10 having the functional configuration of FIG.
The structured data generation process is started when an operation to instruct the start of the structured data generation process is performed via input unit 119 .

In step S11, the UI display control unit 51 accepts the selection of data of the history of the activity to be subjected to the structured data generation process.
In step S12, the activity determination unit 53 determines the type of activity performed by the user based on the data on the type of activity included in the activity history data.

In step S13, the video attribute setting unit 54 sets video attributes (position of the focus point (latitude, longitude, altitude), viewpoint movement speed, viewpoint altitude, viewpoint direction (direction the line of sight is facing), viewpoint horizontal angle (depression angle), actions during various events, etc.) according to the type of activity performed by the user. At this time, the video attribute setting unit 54 sets the video attributes by reading the video attributes corresponding to the type of activity from the video attribute table.
In step S 14 , the parameter correction receiving unit 55 receives a user's operation for correcting the parameters representing the attributes of the moving image set by the moving image attribute setting unit 54 .

In step S15, the structured data generation unit 56 generates structured data for outputting location information based on the attributes of the video set in the activity history data, for displaying data on the movement history (location information) in the activity and content related to the movement history.
In step S<b>16 , the structured data registration unit 57 uploads the structured data for outputting position information generated by the structured data generation unit 56 to the cloud storage server 50 via the smartphone 30 .
After step S16, the structured data analysis process ends.

[Structured Data Analysis Processing]
FIG. 16 is a flowchart illustrating the flow of structured data analysis processing executed by the map information providing server 40 having the functional configuration of FIG.
The structured data analysis process is started when an operation to instruct the start of the structured data analysis process is performed via the input unit 216 or the communication unit 219 .

In step S21, the structured data receiving unit 151 receives the structured data for outputting position information to be processed from the PC 60 together with a request to play back a moving image.
In step S22, the structured data execution unit 152 interprets the structured data for outputting location information received by the structured data reception unit 151. This identifies the address of the related content (such as the URL of the storage area in the cloud storage server 50).

In step S23, the content acquisition unit 153 acquires various contents from the addresses of the related content identified by the structured data execution unit 152 by interpreting the structured data for outputting location information, and outputs the acquired content to the structured data execution unit 152.
In step S24, the structured data execution unit 152 executes the structured data for outputting positional information, thereby generating video data for displaying map information and related content while moving the viewpoint position described in the structured data for outputting positional information in chronological order.

In step S25, the structured data execution unit 152 transmits the generated video data to the PC 60 which is the transmission source of the structured data for outputting position information.
In addition, when the structured data execution unit 152 transmits video data to the PC 60, it is possible to either transmit the entire generated video data all at once, or to divide the video data into small partial data and transmit them.
After step S25, the structured data analysis process is repeated.

[Storage service processing]
FIG. 17 is a flowchart illustrating the flow of a storage service process executed by the cloud storage server 50 having the functional configuration of FIG.
The storage service process is started when an operation to instruct the start of the storage service process is performed via the input unit 216 or the communication unit 219 .

In step S31, the structured data receiving unit 251 determines whether or not structured data for outputting position information has been received.
If the structured data for outputting position information has not been received, the result of the determination in step S31 is NO, and the process proceeds to step S33.
On the other hand, if structured data for outputting position information has been received, a YES determination is made in step S31, and the process proceeds to step S32.

In step S32, the data management unit 255 stores the structured data for outputting position information received by the structured data receiving unit 251 in the structured data storage unit 271 in association with the identification information of the user who transmitted the structured data.
In step S33, the structured data transmission unit 252 determines whether or not a transmission request for structured data for outputting location information has been received. The transmission request for structured data for outputting location information includes the user's identification information and the identification information of the structured data for outputting location information that is requested to be transmitted.

If a request for transmitting structured data for outputting position information has not been received, the result of the determination in step S33 is NO, and the process proceeds to step S36.
If a request for transmitting structured data for outputting position information has been received, a YES determination is made in step S33, and the process proceeds to step S34.
In step S34, the data management unit 255 reads out from the structured data storage unit 271, from among the structured data for outputting location information associated with the user identification information indicated in the transmission request, the structured data for outputting location information that corresponds to the identification information indicated in the transmission request.

In step S35, the structured data transmission unit 252 transmits the read structured data for outputting position information to the device that is the source of the transmission request.
In step S36, the content receiving unit 253 determines whether or not the content has been received.
If the content has not been received, the result of the determination in step S36 is NO, and the process proceeds to step S38.
On the other hand, if the content has been received, the determination in step S36 is YES, and the process proceeds to step S37.

In step S37, the data management section 255 stores the content received by the content receiving section 253 in the related content storage section 272 in association with the identification information of the user who transmitted the content.
In step S38, the content transmission unit 254 determines whether or not a content transmission request has been received. The content transmission request includes user identification information and identification information of the content that is requested to be transmitted.
If a content transmission request has not been received, the result of step S38 is NO, and the storage service process is repeated.
If a content transmission request has been received, then a YES determination is made in step S38, and the process proceeds to step S39.

In step S39, the data management unit 255 reads out from the related content storage unit 272 the content corresponding to the identification information indicated in the transmission request from the content associated with the user identification information indicated in the transmission request.
In step S40, the content transmission unit 254 transmits the read content to the device that has sent the transmission request.
After step S40, the storage service process is repeated.

[Video playback processing]
FIG. 18 is a flowchart illustrating the flow of a video playback process executed by the PC 60 having the functional configuration of FIG.
The video playback process is started when an operation to instruct the start of the video playback process is performed via the input unit 119 .

In step S51, the structured data acquisition unit 351 acquires the structured data for outputting location information by transmitting a request for transmitting the structured data for outputting location information instructed by the user to play a video to the cloud storage server 50. The acquired structured data for outputting location information is stored in the structured data storage unit 371.
In step S52, the video playback request unit 352 transmits a request to play the video to the map information providing server 40 together with the structured data for outputting position information for which playback of the video has been instructed by the user.

In step S53, the video playback unit 353 receives data of a video based on the structured data for outputting position information transmitted from the map information providing server 40.
In step S54, the video playback unit 353 plays the received video data on the display of the output unit 217.
After step S54, the video playback process ends.

Through such processing, in the information processing system 1, structured data for outputting position information is automatically generated from the data of the history of the activity, based on the attributes of the video that are set according to the type of activity.
Then, based on the structured data for outputting position information, the map information providing server 40 generates data of a video that traces the user's movement trajectory and moves on the map. The video data generated at this time includes content related to each position in the movement history, such as markers, photos, comments, etc., and these contents and videos are displayed in a display form suitable for the type of activity.
Therefore, while it is extremely cumbersome for a user to manually set video attributes for movement history data and generate structured data for outputting location information, the presenter can easily set up the presentation of a video that follows the movement trajectory.
That is, it is possible to display a variety of additional information when playing back the moving image trajectory.

[Modification 1]
In the above-described embodiment, when setting video attributes for structured data for outputting location information, video attributes that are preset in accordance with the type of activity are automatically set, but this is not limited to this.
For example, the attributes of a video may be set based on the magnitude, degree of change, or rate of change of predetermined information in a movement trajectory, etc. As an example, the attributes of a video at a position corresponding to the duration of stay in the movement history, the number of captured images, the size (volume) of the captured images, the number of comments, the magnitude of sensor information, the degree of change, or rate of change (highest temperature, highest altitude, fastest wind speed, rate of change in heart rate, etc.) can be set.
This makes it possible to display characteristic positions in the user's movement history in a identifiable manner in the video.

[Modification 2]
In the above-described embodiment, when a user manually sets attributes of a video for structured data for outputting positional information, the attributes of the video may be set by setting a category (evaluation information) for the position to be set. In this case, for example, categories are set as ranks representing the importance of the position, and video attributes corresponding to each category are set in advance. Then, when any category is set for a specific position, the attributes of the video corresponding to that category are set as the attributes of the video at that specific position.
This allows the user to manually set the attributes of a moving image with a simpler operation.

[Modification 3]
In the above embodiment, the content related to the movement history is described as being content that is manually associated by the user by inputting the address of a storage area, etc., but this is not limited thereto.
That is, when associating a movement history with a content, the association method can take various forms.
As an example, the wrist terminal 10 can function as a remote controller for the photographing device 20. In this case, the wrist terminal 10 can receive and display a live view image from the photographing device 20. When a user instructs the wrist terminal 10 to save an image or instructs the photographing device 20 to take a picture, the wrist terminal 10 records an image (such as a thumbnail image) based on the live view image. Then, based on the recording date and time of the image based on the live view image, it becomes possible to associate a specific position in the movement history with the recorded live view image.

As another example, when an instruction to capture an image is given to the imaging device 20, the imaging device 20 can transmit information indicating the capture of an image to the wrist terminal 10, thereby making it possible for the wrist terminal 10 to detect the capture of an image on the imaging device 20. The wrist terminal 10 records the capture date and time in association with the movement history. Then, when structured data for outputting position information is generated, the wrist terminal 10 can identify and associate the image by comparing the capture date and time of the image stored in a specified memory area or the like with the capture date and time associated with the movement history.

As yet another example, the wrist terminal 10 can compare the date, time, and location of the movement history with the shooting date, time, and location of a photograph stored in a specified memory area, etc., and automatically associate content with matching date, time, and location with the movement history.
Content other than photographs can also be associated with movement history in these forms.
This enables the user to easily associate appropriate content with the movement history without having to manually associate the movement history with the content.

[Modification 4]
In the above-described embodiment, when a video tracing a movement history is displayed, content related to the activity history data is displayed based on the video attributes set for each content, but this is not limited to this.
For example, when multiple pieces of content are associated with the same location or nearby locations (e.g., within a few tens of meters), corrections may be made to the attributes of the video that have been set so that the multiple pieces of content as a whole can be displayed more appropriately.
As an example, when mountain climbing, if multiple photos are associated with the same location (e.g., the summit), one of the photos can be selected and displayed, while the other photos can be hidden, or the display time of each photo can be shortened.
In addition, when automatically setting a marker at a specific location depending on the time spent at that location, it is possible to control the system to avoid setting a marker at that location if the action radius per unit time is less than a threshold value.

The wrist terminal 10 configured as above includes a structured data generating unit 56 .
The structured data generation unit 56 generates structured data that defines the display of a video that follows the movement trajectory within a map display provided as map information, based on associated information related to a specified position on the movement trajectory, display control information of a video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory.
The movement trajectory corresponds to the history of a predetermined action of the user.
The structured data generating unit 56 generates structured data so as to display a video in the display control information in accordance with the content of a predetermined action of the user.
This makes it possible to display a video that follows a movement trajectory corresponding to a behavior history while displaying information related to the movement trajectory in accordance with the display control information by executing the structured data.
Therefore, when the movement trajectory is reproduced, it is possible to display the movement trajectory with various information added thereto.

The structured data generation unit 56 associates related information relating to a specified position in the movement trajectory with the specified position in the movement trajectory information, and generates structured data that defines the display of the related information at a specified position in the video tracing the movement trajectory in a display form based on the display control information.
This makes it possible to display a video that follows the movement trajectory while providing an effect in which information related to a given position on the movement trajectory is displayed in accordance with a given display control.

The structured data defines at least one of the position of the attention point, information about the viewpoint, and an action at the time of a preset event.
This makes it possible to define various types of moving images that follow a movement trajectory.

The structured data generating unit 56 generates structured data so as to display a video in the display control information in accordance with the type of a predetermined action of the user.
This makes it possible to display a moving image tracing a movement trajectory in an appropriate display form according to the type of action performed by the user.

The structured data generating unit 56 changes the display control information in accordance with the content of a predetermined action of the user, and generates structured data.
This makes it possible to change the display control information so that a moving image tracing a movement trajectory is displayed in an appropriate manner in accordance with the content of the action performed by the user.

The display control information includes information for controlling at least one of the display time of the related information, the change in the display size, and the expression method in display.
This makes it possible to display related information in a variety of display formats.

The wrist terminal 10 also includes a parameter correction receiving unit 55 .
The parameter correction receiving unit 55 corrects the display control information.
This makes it possible to correct the set display control information in accordance with a user request or the like.

The structured data generating unit 56 automatically acquires preset display control information, or acquires display control information input by the user.
This makes it possible to set the display control information without the user having to make any input, and to set the display control information flexibly in response to the user's request.

The structured data generating unit 56 sets the display control information based on at least one of the size, degree of change, and rate of change of predetermined information in the movement trajectory.
This makes it possible to display characteristic positions on the movement trajectory in a identifiable manner in the video.

The display control information is classified into predetermined categories, and the control content is determined for each classified category.
The structured data generating unit 56 sets the display form of the video that follows the movement trajectory based on the category set for the movement trajectory.
This allows the user to manually set the display control information with a simpler operation.

The structured data generating unit 56 generates structured data that displays related information so as not to overlap with the movement trajectory.
This makes it easy to know which position on the movement trajectory the associated information corresponds to.

The associated information includes captured images, images acquired from other devices, comments, markers, and information from sensors.
This makes it possible to display a variety of related information in association with the movement trajectory.

The map information providing server 40 configured as above further includes a structured data execution unit 152 .
The structured data execution unit 152 executes structured data that defines the display of a video that follows a movement trajectory within a map display provided as map information, the structured data being created based on associated information related to a specific position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory.
The movement trajectory corresponds to the history of a predetermined action of the user.
The structured data execution unit 152 executes the video in accordance with the display control information in response to the content of the predetermined action of the user.
This makes it possible to display a video that follows a movement trajectory corresponding to a behavior history while displaying information related to the movement trajectory in accordance with the display control information.
Therefore, when the movement trajectory is reproduced, it is possible to display the movement trajectory with various information added thereto.

The PC 60 configured as above further includes a video playback unit 353 .
The video playback unit 353 acquires and displays video data resulting from the execution of structured data that defines the display of a video that follows the movement trajectory within a map display provided as map information, based on related information related to a specific position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory.
The movement trajectory corresponds to the history of a predetermined action of the user.
The video playback unit 353 displays the video executed according to the display control information in accordance with the content of a predetermined action by the user.
This makes it possible to display a video that follows a movement trajectory corresponding to a behavior history while displaying information related to the movement trajectory in accordance with the display control information.
Therefore, when the movement trajectory is reproduced, it is possible to display the movement trajectory with various information added thereto.

The present invention is not limited to the above-described embodiment, and any modifications or improvements that achieve the object of the present invention are included in the present invention.

In the above embodiment, when the image capturing device 20 captures images during an activity, the captured image data is uploaded to the cloud storage server 50 via the smartphone 30, but this is not limited to the above. In other words, if the image data captured by the image capturing device 20 can be used for video playback processing on the PC 60, the captured images may be stored in a device other than the cloud storage server 50.

In the above embodiment, the wrist terminal 10 acquires data on movement history and generates structured data for outputting location information, the map information providing server 40 interprets and executes the structured data for outputting location information, and the PC 60 plays videos, but this is not limited to the above. In other words, these processes can be executed by a single information processing device, or can be arbitrarily shared and executed by multiple information processing devices.

In addition, in the above-described embodiment, the content related to the movement history can include various types of information such as captured images (photographs), images acquired from other devices via the network 70, etc., comments, markers, and information detected by sensors (temperature, weather (humidity or air pressure, etc.), altitude, wind speed, and the user's biometric information (body temperature, heart rate)).

In addition, in the above-described embodiment, the presentation format of the content can be defined in the video attributes, such as the display time of the content, changes in the display size, and the method of expression in the display (rotation, blinking, semi-transparent display, etc.).

In addition, the structured data for outputting location information generated in the above-mentioned embodiment can be output, for example, as data in KLM (Keyhole Markup Language) format.

In addition, in the above-described embodiment, the information processing devices 100, 200 to which the present invention is applied have been described as examples of a wrist terminal 10, a photographing device 20, a smartphone 30, a map information providing server 40, a cloud storage server 50, or a PC 60, but are not limited to these.
For example, the present invention can be applied to electronic devices in general that have a function of generating and processing structured data, a function of interpreting and executing structured data, or a function of playing videos based on structured data. Specifically, the present invention can be applied to notebook personal computers, printers, television receivers, video cameras, portable navigation devices, mobile phones, smartphones, portable game consoles, etc.

The above-described series of processes can be executed by hardware or software.
In other words, the functional configurations of Figures 4, 7, and 9 are merely examples and are not particularly limited. That is, it is sufficient for the information processing devices 100 and 200 to have a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the examples of Figures 4, 7, and 9.
Furthermore, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination of both.
The functional configuration in this embodiment is realized by a processor that executes arithmetic processing, and processors that can be used in this embodiment include those configured by various processing devices alone, such as a single processor, a multiprocessor, and a multicore processor, as well as those in which these various processing devices are combined with processing circuits, such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

When the series of processes is executed by software, the program constituting the software is installed into a computer or the like from a network or a recording medium.
The computer may be a computer built into dedicated hardware, or may be a computer capable of executing various functions by installing various programs, such as a general-purpose personal computer.

The recording medium containing such a program may be constituted not only by the removable media 131, 231 in Figs. 2 and 3 that are distributed separately from the device main body in order to provide the user with the program, but also by a recording medium provided to the user in a state in which it is pre-installed in the device main body. The removable media 131, 231 may be constituted, for example, by a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. The optical disk may be constituted, for example, by a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or a Blu-ray (registered trademark) Disc. The magneto-optical disk may be constituted, for example, by an MD (Mini-Disk). The recording medium provided to the user in a state where it is pre-installed in the device body is composed of, for example, the ROM 112, 212 in Figures 2 and 3 on which the program is recorded, or the hard disk included in the storage unit 121, 218 in Figures 2 and 3.

In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually.
In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc.

Although several embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

The invention as described in the claims of the present application as originally filed is set forth below.
[Appendix 1]
a structured data generating means for generating structured data defining a display of the video tracing the movement trajectory within a map display provided as map information, based on related information relating to a predetermined position on the movement trajectory, display control information of the video tracing the movement trajectory, and movement trajectory information representing the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
The information processing device is characterized in that the structured data generation means generates the structured data so as to cause the video to be displayed in the display control information in accordance with the content of the predetermined action of the user.
[Appendix 2]
The information processing device described in Appendix 1, characterized in that the structured data generation means associates related information relating to a predetermined position in the movement trajectory with the predetermined position in the movement trajectory information, and generates the structured data that defines the display of the related information at the predetermined position in a video that follows the movement trajectory in a display form based on the display control information.
[Appendix 3]
3. The information processing device according to claim 1, wherein the structured data defines at least one of a position of a point of interest, information related to a viewpoint, and an action at the time of a preset event.
[Appendix 4]
The information processing device described in any one of appendices 1 to 3, characterized in that the structured data generation means generates the structured data so as to display the video in the display control information in accordance with the type of the specified behavior of the user.
[Appendix 5]
The information processing device according to any one of claims 1 to 4, wherein the structured data generation means generates the structured data by changing the display control information in accordance with the content of the specified action of the user.
[Appendix 6]
6. The information processing device according to claim 1, wherein the display control information includes information for controlling at least one of a display time, a change in display size, and a method of expression in display of the related information.
[Appendix 7]
7. The information processing device according to claim 1, further comprising a correction unit that corrects the display control information.
[Appendix 8]
The information processing device according to any one of claims 1 to 7, wherein the structured data generation means automatically acquires the display control information that has been set in advance, or acquires the display control information that has been input by a user.
[Appendix 9]
The information processing device described in any one of appendix 1 to 8, characterized in that the structured data generation means sets the display control information based on at least one of the size, degree of change, or rate of change of specified information in the movement trajectory.
[Appendix 10]
The display control information is classified into predetermined categories, and control content is determined for each of the classified categories,
The information processing device according to any one of claims 1 to 9, wherein the structured data generation means sets a display form of a video that follows the movement trajectory based on the category set for the movement trajectory.
[Appendix 11]
The information processing device according to any one of claims 1 to 10, wherein the structured data generation means generates the structured data that displays the related information so as not to overlap with the movement trajectory.
[Appendix 12]
12. The information processing device according to claim 1, wherein the related information includes captured images, images obtained from other devices, comments, markers, and information from sensors.
[Appendix 13]
a structured data execution means for executing structured data that defines a display of the video that follows the movement trajectory in a map display provided as map information, the structured data being created based on related information related to a predetermined position in the movement trajectory, display control information of the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
The information processing device, wherein the structured data execution means executes the video in accordance with the display control information in response to the content of the predetermined action of the user.
[Appendix 14]
a video display means for acquiring and displaying video data resulting from execution of structured data that defines display of the video that follows the movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
The information processing apparatus, wherein the video display means displays the video executed according to the display control information in accordance with the content of the predetermined action of the user.
[Appendix 15]
a structured data generating step of generating structured data defining a display of the video following the movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information of the video following the movement trajectory, and movement trajectory information representing the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
An information processing method characterized in that, in the structured data generation step, the structured data is generated so as to cause the video to be displayed in the display control information in accordance with the content of the specified action of the user.
[Appendix 16]
a structured data execution step of executing structured data that defines display of the video following the movement trajectory in a map display provided as map information, the structured data being created based on related information related to a predetermined position in the movement trajectory, display control information of the video following the movement trajectory, and movement trajectory information representing the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
The information processing method, wherein in the structured data executing step, the video is executed in accordance with the display control information in response to the content of the predetermined action of the user.
[Appendix 17]
a video display step of acquiring and displaying video data resulting from execution of structured data that defines display of the video that follows the movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information of the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
The information processing method, wherein in the moving image display step, the moving image executed according to the display control information is displayed in accordance with the content of the predetermined action of the user.
[Appendix 18]
On the computer,
a structured data generation function that generates structured data that defines a display of the video that follows the movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information of the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
The structured data generation function is a program characterized by generating the structured data so as to cause the video to be displayed in the display control information in accordance with the content of the specified action of the user.
[Appendix 19]
On the computer,
A structured data execution function is realized that executes structured data that defines the display of the video that follows the movement trajectory within a map display provided as map information, the structured data being created based on related information related to a predetermined position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory,
the movement trajectory corresponds to a history of a predetermined action of the user,
The structured data execution function is a program characterized in that the video is executed in accordance with the display control information depending on the content of the specified action of the user.
[Appendix 20]
On the computer,
a video display function is realized that acquires and displays video data resulting from execution of structured data that defines the display of the video that follows the movement trajectory within a map display provided as map information, based on related information related to a predetermined position on the movement trajectory, display control information for the video that follows the movement trajectory, and movement trajectory information that represents the movement trajectory;
the movement trajectory corresponds to a history of a predetermined action of the user,
The video display function is a program that displays the video executed according to the display control information in response to the content of the predetermined action of the user.

1: Information processing system, 10: Wrist terminal, 20: Photographing device, 30: Smartphone, 40: Map information providing server, 50: Cloud storage server, 60: PC, 70: Network, 111, 211: CPU, 112, 212: ROM, 113, 213: RAM, 114, 214: Bus, 115, 215: Input/output interface, 116: GPS unit, 117: Photographing unit, 118: Sensor unit, 119, 216: Input unit, 120, 217: Output unit, 121, 218: Memory unit, 122, 219: Communication unit, 123, 220: Drive, 131, 231: Removable media, 51: User interface Interface display control unit (UI display control unit), 52... history acquisition unit, 53... activity determination unit, 54... video attribute setting unit, 55... parameter correction reception unit, 56... structured data generation unit, 57... structured data registration unit, 71... history data storage unit, 151, 251... structured data reception unit, 152... structured data execution unit, 153... content acquisition unit, 252... structured data transmission unit, 253... content reception unit, 254... content transmission unit, 255... data management unit, 271, 371... structured data storage unit, 272... related content storage unit, 351... structured data acquisition unit, 352... video playback request unit, 353... video playback unit

Claims (12)

a structured data generating means for generating structured data for displaying a trajectory of the movement history and content related to the movement history within a map display provided as map information, based on data of a history of an activity including a movement history of a user and a type of the activity;
The structured data is an information processing device characterized in that an event related to the display of the content is set at a predetermined position on the trajectory of the movement history, and an operation at the time of the event, including a time of change in the movement speed of a point of interest tracing the trajectory of the movement history at the predetermined position or whether or not to zoom in, is set based on the type of activity. 2 . The information processing apparatus according to claim 1 , wherein the structured data generating means generates the structured data based on attributes of a video according to the type of the activity. The information processing device according to claim 2, characterized in that the structured data generation means determines the type of activity performed by the user based on data on the type of activity included in the activity history data, and sets attributes of the video according to the type of activity. 4. The information processing apparatus according to claim 1, further comprising an activity receiving unit that receives an input of the type of the activity. 4. The information processing device according to claim 3, wherein the attributes of the video are defined as at least one of the position of a focus point along the trajectory of the movement history, information related to a viewpoint, and an action at the time of the set event. 6. The information processing apparatus according to claim 1, wherein the structured data generating means generates the structured data so that a marker according to the type of the content is displayed at the predetermined position. 6. The information processing apparatus according to claim 5 , wherein the structured data is generated so that, when the attention point reaches a position where the event is set, an action of the set event is executed. 6. The information processing device according to claim 5, characterized in that the information regarding the viewpoint is defined as a moving speed that is a constant multiple of the actual moving speed in the activity, a selected specific speed, or a speed at which the entire process is moved in a preset time. 6. The information processing apparatus according to claim 5 , wherein the information regarding the viewpoint defines an altitude of the viewpoint according to the type of the activity. 6. The information processing apparatus according to claim 5, wherein an action corresponding to the content is defined as an action at the time of an event. a structured data generating step of generating structured data for displaying a trajectory of the movement history and content related to the movement history within a map display provided as map information, based on data of a history of an activity including a movement history of a user and a type of the activity;
The information processing method is characterized in that the structured data sets an event related to the display of the content at a predetermined position on the trajectory of the movement history, and sets, based on the type of activity, an operation at the time of the event, including the time of change in the movement speed of a point of interest tracing the trajectory of the movement history at the predetermined position or whether or not to zoom in. On the computer,
A structured data generation function is provided for generating structured data for displaying a trajectory of the movement history and content related to the movement history within a map display provided as map information, based on data of a history of activity including a user's movement history and a type of the activity;
The structured data is a program characterized in that an event related to the display of the content is set at a specific position on the trajectory of the movement history, and an action at the time of the event is set based on the type of activity, including the time of change in the movement speed of a point of interest tracing the trajectory of the movement history at the specific position or whether or not to zoom in.