EP1185142A2

EP1185142A2 - System for viewing visual content

Info

Publication number: EP1185142A2
Application number: EP01115771A
Authority: EP
Inventors: Edward G. Newman
Original assignee: Xybernaut Corp
Current assignee: Xybernaut Corp
Priority date: 2000-08-30
Filing date: 2001-07-10
Publication date: 2002-03-06
Also published as: JP2002153684A; EP1185142A3; CA2350015A1

Abstract

The invention is basically a system for delivering closed caption text to attendees of movies or other events. It comprises a head mounted display 103 which can be used at motion picture theaters for hearing impaired persons and persons who don't speak the language that the original movie dialog is being shown in and also for distributing broadcast information to the same persons at other events. The main components are a device 103 comprising either a suitable monocular or binocular display, including a head mounted display which is worn by viewers of movies or attendees of events, a central router/processor 100, and a transmission protocol 104. Closed caption or representative text is projected onto the screen of the display 103, synchronized with the dialog of the movie or broadcast, and in a manner that permits the person wearing it to view the action on the screen, while it is completely discrete to those who are not using the display 103. For persons who speak a foreign language there will be an interface 116 for choosing the language of the closed caption text. The device will either receive its signal information wirelessly or will plug into a connector built into the seat or other nearby structure. A power supply mechanism will either be integral or attached to the device or will also be supplied by a hardwire connection 107.

Description

The invention relates to a system for viewing visual content according to the features of claim 1.
The invention is generally related to a method for viewing closed captioning during movies, displayed or available to be displayed, at the theater or on TV or other display devices. It includes a head mounted display for watching the closed caption text or for watching subtitled information for movies which are being shown in a language foreign to the viewer of the movie or for the benefit of those who are hearing impaired. This application is related to concurrently filed application entitled, "System for delivering synchronized audio content to viewers of movies," and pending application, "Multimedia I/O Interface Device for use at Entertainment Events," both commonly assigned.
The world motion picture industry has developed into a multibillion dollar a year business. Centered in the US, the industry is creating hundreds of feature films per year which generate billions of dollars annually at the box office. Due to the combinations and mergers which now characterize the motion picture industry, movies are being made by large diversified media companies which also own consumer product companies. As a result, motion pictures are more frequently being used as vehicles for advertising products and promoting the parent company's suite of goods. Even before movies start in the theater, it is now common to see commercials being played, and there are always short trailers describing other upcoming or currently released movies. Thus, it is important to these media companies to maximize their return on investment by making the movies available to the largest possible viewing audience. There are tens of thousands of movie showings in the U.S. alone each day which creates an enormous potential for market exposure. However, at the present there are several significant segments of the market which are not being addressed by the motion picture industry. First of all, is the hearing impaired segment. According to the National Association of the Deaf, a U.S. based non-profit organization, there are 28 million deaf or hard of hearing persons in the U.S. alone. This is at least 10% of the nation's population and is likely to be a consistent or higher percentage in other countries as well. It includes those who are completely deaf and those with selective deafness who have difficulty hearing dialog over other ambient sounds and noises. Currently, the motion picture industry creates movies with no provisions to accommodate those people. Deaf persons are forced to wait until the movie comes out on video and to get a copy with the dialog in subtitle format at the bottom of the picture. This text corresponds in real time to all verbal dialog in the movie.
A problem with this is that even when the movies do come out on video cassette, most are not available with this feature. Thus, deaf people are discouraged from or unable to watch movies either in their first run or at home.
A potential solution is to use subtitled text in the movie theater so that hearing impaired people can read the dialog while others listen to it. This is being done on a very small scale in a few cities. There are currently six theaters in six different cities which have dedicated screens for open captions which are permanently displayed and tens of others which show captioned versions on certain days of the month. The open captions appear as printed text at the bottom of the screen simultaneous to the dialog. The reason this captioning hasn't become more pervasive is that those who are not hearing impaired, who make up a much larger percentage of the population, do not wish to see this text being displayed as it visually detracts from the image on the screen. Also, it is not cost effective to broadcast two versions of the movie, one with subtitles and one without them when only ten percent or less of the box office ticket is hearing impaired. Hearing impaired people with selective deafness can wait until the few first run movies that are captioned come out on video tape and comfortably watch them at home where they can control the volume. However, they are generally discouraged from attending the theater because it is difficult for them to discriminate the dialog over the ambient sound and there are currently no provisions which allow for individualized amplification of the dialog in the theater.
Recently, two class action lawsuits have been filed in the U.S., one in Oregon and the other in the District of Columbia, against movie theater companies. The suits assert that these companies have not made sufficient efforts to accommodate hearing impaired persons at the theater. The lawsuit states that the theaters failed to make "reasonable modification" in their practices, and failed to take necessary steps to insure that deaf people are not excluded from or denied first run movies. Thus, there is a clear and present need for a solution to mitigate this problem, a solution which will be cost effective and robust, but which will not alienate current movie goers. It is in the interest of the film makers, the theater owners, and the viewing public to come up with an effective solution to this problem.
The second significant demographic currently being ignored by the motion picture industry is the non-English speaking one. Thus, the same problem of non-accommodation exists with respect to foreign language dialog. A significant percentage of persons living in the U.S. do not speak English at least as a first language. According to the 1990 census, Hispanic Americans accounted for 8.8% of the U.S. population. By the 2000 census, this percentage will likely be well over 10%. Asian Americans such as Chinese, Vietnamese, and Koreans also make up an important percentage of the national demographic. Much of the rapid growth of each of these populations is due to the recent immigration of persons who speak little or no English. These people are discouraged from seeing first run movies at the theater because they are unable to understand the English dialog. However, the media industry is recognizing the market power of this demographic as evidenced by the growth of Hispanic television and radio networks and Asian television networks. This problem could be potentially solved by writing subtitled translation of the dialog on the screen; however, this still suffers from the limitation that it is visually displeasing to those who do not need the subtitled text. Also, it allows for only two simultaneous languages to be broadcast, the one the sound track is recorded in and the subtitled one.
Another demographic is native citizens who wish to watch foreign films which are shown in the U.S. In larger metropolitan cities, popular foreign movies are frequently played at theaters in their original language either without subtitles or with permanent subtitles. It would be desirable for persons attending these films to designate a language to view the movies in.
Thus, there exists a need for a cost effective system for allowing viewers of movies to view and/or listen to dialog, in text or symbolic format, which corresponds in real time or near-real time to the dialog of the movie, in a language of the viewer's choice.
It is an object of this invention to create a system which is devoid of the shortcomings described above.
It is another object of this invention to create a system for viewing text or symbolic representation of text corresponding to real time dialog of motion pictures.
It is an additional object of this invention to create a system which allows viewers to view subtitled text of movies and other media content such as sporting events, coverage of live events, etc.
Another object of this invention is to create a system which will be in synchronization with active dialog, and will display that dialog to viewers in text or symbolic format.
Yet another object of this invention is to create a system which will provide individualized captioning for viewers of motion pictures while at the movie theater or elsewhere.
Still yet another object of this invention is to provide a system which is worn on the head like glasses.
An additional object of this invention is to provide a system which will display dialog to the viewer without obstructing his view of the action on the screen.
Yet another additional object of this invention is to provide a system which allows users to selectively choose the language of the displayed dialog.
These and additional objects of this invention are accomplished generally through a head worn device, or other suitable display, for viewing closed caption text and optionally listening to dialog while watching motion pictures at the movie theater or while attending other public events, or at home via a television receiver.
Additional advantages and characteristics result from the following description of several embodiments of the invention in connection with the drawings. In the drawings are shown:

Fig. 1: illustrates a theater embodiment with all the system components;
Fig. 2: illustrates an exemplary device in a wired connection embodiment;
Fig. 3: illustrates an exemplary device in a wireless connection embodiment
Fig. 4: illustrates an event-based embodiment of the system; and
Fig. 5: illustrates a close-up of the control module.

Figure 1 illustrates a system level diagram of the fundamental components. The film source 101 feeds through a communications link 102 to a central router/processor 100. The router/processor 100 will at least handle the broadcast of the closed caption text. Optionally, it may perform on-the-fly conversion of audio dialog to closed captioning and even translation to other languages if necessary. The output of this router will be transmitted through communications link 104, which could be a wired or wireless link, to the individual user viewer means 103, which can include head mounted displays, flat panel display, neck hung display, wrist worn display, or any other suitable display. It may be desirable to have a lighted display in some instances. "Viewing means" includes any suitable user supported display in the context of the present disclosure and claims.
Figures 2 and 3 illustrate wired and wireless headset or viewing means embodiments whereby the headset 103 is configured as a set of glasses. The headset glasses 103 use power/signal wire 106 to connect to a connector in or near a theater seat, via connector 107. In the data stream is control module 105, used to affect various controls available to the head mounted display 103. In figure 3 the connector 107 is replaced with a combined battery pack and wireless receiver 108 so that the device can operate in a wire free mode, thus eliminating the need for connector 107.
Figure 4 illustrates the system in the context of an event, other than a movie, where persons may be moving around within a certain range. While similar to the movie theater embodiment, the devices in this case will preferably be wireless so that persons can use them while moving about the event. The event management computer and transmitter 109, routes and sends out any audio messages to the attendees of the events, via wireless communication link 110, to be displayed on the user headset display 103. The computer 109 can also perform translation of messages if desired. There are a plurality of head mounted user devices or viewer means 103, which receive the messages and displays them to the user in his desired language.
Figure 5 illustrates an exemplary control module 105. Brightness control 113, focus control 114 and position control 115 are shown. There is an optional language selection interface and control 116 which allows users to designate the language of the closed caption text which will appear on their display units 103. There may also be a concessions interface 111 depending upon where the device is used, which contains item selection buttons 112 allowing users to select specific concession items to be delivered to their location. This will be particularly useful if the device is used in a movie theater or other viewing locations, as it will allow persons to order concession items without having to leave their seat.
The invention is directed to a method of delivering and viewing closed caption or subtitled or symbolic text dialog in movie theaters or at other locations or events. It is comprised essentially of a binocular, monocular, or eyeglass-type head mounted display 103, a central router and processor 100, and a transmission mechanism and protocol 104. The enclosures of concurrently filed application entitled, "System for delivering synchronized audio content to viewers of movies" and pending application entitled "Multimedia I/O Interface Device for use at Entertainment Events" are hereby incorporated by reference into the disclosure of this invention. The display 103 can be in the form of a pair of electronic glasses or visors with an integral screen. The screen or viewing means allows viewers to look through it so that they can view the real time movie, but also displays text which is correlated to the current dialog occurring in the motion picture or at the event, in a manner analogous to the heads up displays used by pilots in military aircraft who need to be able to read textual output while also looking ahead. This will allow deaf persons to go to movie theaters and follow the dialog in a manner that is transparent to those who are not hearing impaired. By "movie theaters" is meant any location where a movie can be seen. The display 103 can also have a control 116 for selecting the language of either the spoken or subtitled dialog, which will allow viewers who wish to see or listen to the dialog in a language other than the one being broadcast on the screen the ability to do so.
The invention will require the ability to deliver, in real time or near real time, the text content to the viewers' display units so that it will correlate to the action on the screen. Closed captioning technology is well known in the entertainment production business as it comes standard on televisions now to accommodate hearing impaired persons. For this invention to work robustly and accurately, several versions of the dialog will have to be created when the final version of the movie is produced, both in English and other popular languages. The solution to this problem would be efficiently solved by a time or frame based Closed Caption Markup Language, hereinafter CCML © . The beginning and end of the text dialog can be tagged to frames of the film so that it would be seamlessly and synchronously integrated with the dialog in the movie. A simple software protocol could be written to facilitate the creation and positioning of the closed caption text. This CCML© will create a new standard for merging closed captioning to existing movie playback formats. It will also be useful for digital movies because of the similar manner in which digital movies are produced using a series of single frames. Using frame tags, a portion of text can be given stop and start points so that it is synchronized with the audible dialog and action on the screen. This standard can also be used for delivering commercial information, movie trailer dialog, and safety information such as that which is typically broadcast prior to the beginning of a feature presentation. Alternatively, a robust captioning engine, running on the router/processor 100, could be used which could either prior to the movie, simultaneous to the movie or slightly ahead of the movie compile and deliver the translated text. This engine would possess a high quality textual and contextual translation engine in order to perform several language translations on-the-fly. The central router and processor 100 will handle the broadcast of this data to the individual user devices 103. This will be comprised of at least a transmitter and alternatively a server computer to perform the processing on-the-fly or slightly ahead of time generation and/or translation and/or transmission of the closed caption text. As microprocessors become more robust this near real time processing will become more feasible. This could also be used for watching movies at home to either translate or merely close caption the audio dialog. The invention would serve as a preprocessor that interfaces between the video signal source of a movie and the television display.
Once the text is produced and is synchronized with the action in the movie, it merely needs to be delivered to the head mounted display of the viewers. This can be accomplished through one of several embodiments using existing and/or future technologies. These embodiments can be generally segmented into two types, wired and unwired. In a wired embodiment, the device itself has a plug on it that plugs into a connector which is integral to the movie theater seat. Much like airplane headphones, the user merely sits down and plugs them in. The connector will provide both power and content to the device 103. This will eliminate the need for an integral power supply, will enhance reliability of the system, and minimize the cost of the individual devices 103. Also, viewers can purchase the devices either at the theater or elsewhere, reducing the cost to the movie theater owners who merely provide the backbone and interface connection to the viewer's devices 103. The format of the data itself will be either digital or analog, as only fairly narrow bandwidth will be required to carry this textual information. The display 103 itself can be digital or analog as both methods have well known advantages and disadvantages.
In another embodiment, the data can be transmitted wirelessly. This could either be as closed circuit analog TV, RF, or through digital transmission means such as IEEE 802.11, or any other existing or future transmission protocols. One of ordinary skill in the art will understand that the method of transmission is not critical to the invention itself and that various variations and substitutions may be made from the present disclosure without departing in spirit or scope from the essence of the invention. If analog technology is used, a low power closed circuit TV transmitter could be placed in the projector room or elsewhere in the theater servicing the entire room with each language broadcast on a different channel band. If digital transmission is used, a wireless LAN transmitter/- receiver could be placed in the projector room or elsewhere in the theater so that wireless communications could be maintained with individual headsets 103. In this embodiment, the headsets themselves will possess wireless receiving means 108 in order to receive the signal. This could be a separate receiver box which is connected to the headset 103 which could also house the battery 108 for the receiver or a receiver integral to the headset 103, or the seats themselves could contain the receiver or transceiver. If analog transmission means are utilized, then the headset 103 will require an integral analog signal receiver and optionally a tuner. In either case it may be desirable to include a headphone speaker or speakers for those who are not completely deaf, but hearing impaired to the extent that they can not easily hear text dialog without selective amplification, due to other background noise and music in the movie soundtrack.
In an additional embodiment, the device will have an integrated interface 111 for ordering food or beverages from the concession area outside the theater which are delivered to the seat location of the user/wearer of the system. The interface will be comprised of item selection buttons 112. When the users get the devices, upon arrival at the theater, or sometime before that, they can establish a billing account to pay for such items that are ordered while at the movie or event, or they can pay for them at time of delivery. This will have particular benefits if the device derives its power and signal from the seat location as it will enable the person delivering the food to identify the exact location of the person ordering it.
In yet another embodiment, the device 103 can be worn or carried at events or attractions such as sporting events, amusement parks, artistic events, or other publicly or privately attended events where audio information is broadcast aloud. In this manner, promotional/advertisement information, safety information, public service information, and information specific to the event or attraction can be delivered to those who are hearing impaired or those who do not speak the language of the broadcast dialog. This would not only enhance the experience of the attendees but would also allow those who are sponsoring and managing the event or attraction to deliver information to a wider audience further increasing the return on investment for themselves.
As to the power supply, in either wireless embodiment, the power could be supplied via a connector built in to the theater's seat, or the headset 103 itself could have an integral or attached battery supply. As more efficient fuel cells are developed, it will eventually become cost efficient to use batteries. However, at present, drawing the power for the device 103 from the connection in the theater's seats will increase overall system reliability, reduce costs, and eliminate device interrupts due to loss of power.
As to the viewing unit itself, several companies such as Sony Corporation, Shimadzu Corporation, Minolta Corporation, MicroVision, among others, all make small monocular and binocular displays head mounted displays. They are worn comfortably on the head like eyeglasses or headsets and can support analog or digital signal display. The display 103 will optionally have a channel selector 116 which allows the wearer to selectively choose the language he wishes to view or listen to the subtitled text in. It will also optionally have manual focus 114, brightness 113, and positioning 115 controls so that the user can move the text line to a position which optimizes his viewing of the action on the screen. It will allow the wearer to adjust for different viewing distances depending upon where he is seated in the theater so that the glasses create the illusion that the text is projected on the screen. Optionally, the system can automatically detect its distance from the screen and adjust the focus, brightness and position for optimum viewability. It could also attach to, fit over, or extend beyond a viewer's prescription glasses enabling him to wear his glasses if necessary to view the movie screen.
Thus, the disclosed invention overcomes the current deficiencies of the state of the art in the motion picture and entertainment industry. It creates a new standard for selectively delivering dialog to viewers of movies in the theater or persons attending other events, that is invisible to those who are not using it. It provides heretofore unknown flexibility in accommodating those who are either hearing impaired or who do not understand the language the movie is being played in. It also provides a medium for interaction with the facility and/or event management and a platform for commercial transactions.
The preferred and optimumly preferred embodiments of the present invention have been described herein to illustrate the underlying principles of the invention, but it is to be understood that numerous modifications, designs, and alterations may be made without departing from the spirit and scope of this invention.

List of Reference Signs

100: router/processor
101: film source
102: communications link
103: viewer means
104: communications link/protocol
105: control module
106: power/signal wire
107: connector
108: receiver/battery
109: computer/transmitter
110: communication link
111: concessions interface
112: selection buttons
113: brightness control
114: focus control
115: position control
116: language selection interface and control/channel selector

Claims

A system for viewing visual content such as text or symbolic dialog of a movie, especially in a movie theater, comprising:

a display for viewing said visual content for a viewer;

a transmission protocol delivering the textural position of said content to the display of the viewer;

a power mechanism for supplying power to said viewer's display.
A system according to claim 1, characterized in that the display is wearable by the viewer for viewing said visual content.
A system according to claim 1 or 2, characterized by a closed circuit markup language ("CCML") for integrating said text or symbolic dialog into a movie for playing at the movie theater.
A system according to any one of the preceding claims, characterized in that said display is selected from the group consisting of a head mounted display, holographic display, flat panel display, neck hung display, user supported display and mixtures thereof, said display may be illuminated or not.
A system according to any one of the preceding claims, characterized in that said display is a binocular display.
A system according to claims 1 to 4, characterized in that said display is a monocular display.
A system according to any one of claims 4 to 6, characterized in that said display is an eyeglass-type display worn on the head of a person.
A system according to claim 2 to 7, characterized in that said display is operable to fit over or attach to a person's prescription eyeglasses.
A system according to any one of the preceding claims, characterized in that said transmission protocol is selected from the group consisting of wired and wireless transmission protocols.
A system according to any one of the preceding claims, characterized in that the power delivery mechanism includes a power connector integral to a movie seat and is connected via a power cable to the display.
A system according to any one of the preceding claims, characterized in that the power delivery mechanism includes a DC type power storage device integral or attached to the display.
A system according to any one of the preceding claims, characterized in that said display is an analog display.
A system according to claim 10 and 11, characterized in that said display is a digital display.
A system according to any one of the preceding claims, characterized in that said transmission protocol is wired transmission facilitated by an output connector in a movie theater seat.
A system according to any one of the preceding claims, characterized in that said transmission protocol is a wireless transmission protocol.
A system according to any one of the preceding claims, characterized in that means are included in said device for ordering food or beverages to be delivered to the location of the viewer.
A system according to any one of the preceding claims, characterized in that means are included for optimizing the focus of the display in relation to the distance of a viewer from a screen.
A system according to any one of the preceding claims, characterized by delivering audio content related to an event to attendees of an event who are hearing impaired.
A system according to any one of the preceding claims, characterized in that said display includes a control for allowing a viewer to select a desired language for viewing said content.