{"id":3096171,"date":"2024-12-05T13:15:00","date_gmt":"2024-12-05T11:15:00","guid":{"rendered":"http:\/\/nhub.news\/?p=3096171"},"modified":"2024-12-05T20:16:43","modified_gmt":"2024-12-05T18:16:43","slug":"googles-live-captions-can-now-use-ai-gasp-to-make-expressive-captions","status":"publish","type":"post","link":"http:\/\/nhub.news\/ru\/2024\/12\/googles-live-captions-can-now-use-ai-gasp-to-make-expressive-captions\/","title":{"rendered":"Google\u2019s Live Captions can now use AI [gasp] to make \u2018Expressive Captions\u2019"},"content":{"rendered":"<p style=\"text-align: justify;\"><b>Array<\/b><br \/>\nGoogle\u2019s Live captions are becoming richer with new AI-driven \u201cExpressive Captions\u201d that convey more than basic language, including sounds and actions. Google is also bringing Gemini 1.5 to Image Q&#038;A in the Lookout app.<br \/>Live Caption has been a staple of Google\u2019s Pixel lineup since 2019. The feature allows users to insert captions where there are normally none using the phone\u2019s Tensor SoC and onboard processing. When a voice is heard through a video or other media playing audio, the Pixel phone will pick up on that speech and display it as it hears it. It\u2019s useful for a variety of users, especially those who are deaf\/hard of hearing.<br \/>Live Captions are getting an overhauled mode for processing audio more dynamically. Google announced that Expressive Captions would allow users to see the nuanced speech and actions in media through Live Captions using AI on-device. That includes decoding tone, volume, and environmental cues. The change will dynamically reflect the way speech is presented.<br \/>Google gives a couple of examples of how this will work. When someone yells something, that intensity is translated to captions in all caps. If someone were to yell, the caption would reflect the volume. Google\u2019s expressive captions using AI can also decode vocal bursts, such as sighs and groans, detailing the little sounds in between words. Even ambient sounds are represented to fill in the blacks around speech.<br \/>In addition, Google announced that image descriptions can now be read aloud. With that, the company is bringing Gemini 1.5 Pro to the Lookout app \u2013 an app that aids the vision-impaired. The Q&#038;A feature, which allows users to ask questions about an image, will now be a little more capable. An image can be described in a more natural voice via the Gemini model and will be capable of giving more surrounding information beyond a simple description.<br \/>It\u2019s noted that Google\u2019s expressive AI captions are a part of Live Caption, so there is no restriction to which Pixel devices can utilize it. If Live Caption is available, this upgrade will be reflected. Google does note that the feature will not be compatible with phone calls, though that might change over time.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Array Google\u2019s Live captions are becoming richer with new AI-driven \u201cExpressive Captions\u201d that convey more than basic language, including sounds and actions. Google is also bringing Gemini 1.5 to Image Q&#038;A in the Lookout app.Live Caption has been a staple of Google\u2019s Pixel lineup since 2019. The feature allows users to insert captions where there [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3096170,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[90],"tags":[],"_links":{"self":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/posts\/3096171"}],"collection":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/comments?post=3096171"}],"version-history":[{"count":0,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/posts\/3096171\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/media\/3096170"}],"wp:attachment":[{"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/media?parent=3096171"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/categories?post=3096171"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/nhub.news\/ru\/wp-json\/wp\/v2\/tags?post=3096171"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}