#Multimodality
OmniFusion: выходим за границы текста
Кто-то ещё сомневается, что в мире машинного обучения происходит революция? Уверен, мы являемся свидетелями преобразования привычного взаимодействия с данными, поиска информации, да и вообще работы как таковой. Ведь умные ассистенты (ChatGPT, GigaChat, Bard) готовы взять на себя даже самые сложные задачи. Но не всегда возможно сформулировать проблему в виде текстового запроса, иногда требуется информация из других “модальностей” — картинка, звук, 3D и тд. Ниже я разберу какие именно есть способы соединения больших языковых моделей (LLM) с дополнительными форматами данных, а также опишу как устроена наша новая модель OmniFusion.
Kandinsky 3.0 — новая модель генерации изображений по тексту
Без чувства современности художник останется непризнанным. Михаил Пришвин В прошлом году на АI Journey мы представили модель Kandinsky 2.0 — первую диффузионную мультиязычную модель генерации изображений по тексту, которая может генерировать изображения на основе русскоязычного текста. За ней последовали новые версии — Kandinsky 2.1 и Kandinsky 2.2 , которые значительно отличались по качеству и своим возможностям от версии 2.0, и стали для нашей команды серьёзными вехами на пути к достижению лучшего качества генерации. Спустя год после релиза нашей первой диффузионной модели мы представляем новую версию модели генерации изображений по тексту — Kandinsky 3.0! Это результат длительной работы нашей команды, которую мы вели параллельно с разработками версий Kandinsky 2.1 и 2.2. Мы провели много экспериментов по выбору архитектуры и проделали большую работу с данными, чтобы сделать понимание текста и качество генераций лучше, а саму архитектуру — проще и лаконичнее. Также мы сделали нашу модель более «отечественной»: теперь она значительно лучше ориентируется в российском и советском культурном поле. В этой статье я кратко опишу ключевые моменты новой архитектуры, стратегию работы с данными и, конечно, продемонстрирую возможности нашей модели на примере генераций.
https://habr.com/ru/companies/sberbank/articles/775590/
#kandinsky_30 #multimodality #sberai #sberdevices #airi #generative_models #kandinsky #computer_vision #texttoimage #animation
A warm welcome to Falko Helm, who has just started as a PhD candidate at the UKP Lab! 👋 Falko researches #Multimodality & Structure for #TransformerModels. He is also interested in #GraphTheory to handle linked documents. Find more about Falko here: https://github.com/Falko1
Watch the discussion on enhancing active mobility! Experts D. Taylor Reich Carlosfelipe Pardo share insights on the power of open data and technology in this enlightening webinar recording. Data-driven decision-making and advocacy can transform cycling infrastructure.
#opendata #opensource #sustainabletransport #ActiveTransport #ActiveMobility #Cycling #Walking #CommuteByBike #mapping #Multimodality #mobility #OpenStreetMap #VGI #multimodal #mobileapp #webinar
🚴♂️ Join us this Thursday for a deep dive into the world of active mobility! 🚶♀️ Don't miss out on insights from experts Taylor Reich and Carlosfelipe Pardo. Discover the impact of data in shaping our cities. Register now:
https://www.linkedin.com/events/unpackingthepowerofdatainactive7109631505320071168/theater/
#opensource #sustainabletransport #transportation #ActiveTransport #Cycling #Walking #CommuteByBike #MobilityAsAService #mapping #Multimodality #mobility #OpenStreetMap #VGI #GTFS #GBFS #multimodal #webinar
Active mobility is the future, and data is driving the way. Join our webinar with experts Taylor Reich and Carlosfelipe Pardo to explore the intersection of walking, cycling, and data.
#opendata #opensource #sustainabletransport #transportation #ActiveTransport #ActiveMobility #Cycling #Walking #CommuteByBike #digitaldevelopment #Multimodality #mobility #populartransport #OpenStreetMap #VGI
Congratulations, MCO & Brightline! #Orlando International is now the first true U.S. #multiport: https://www.gobrightline.com/press-room/2023/the-countdown-is-on-brightline-orlando-to-officially-launch-service. More about multimodal #airport facilities in our upcoming #ACRP
#research report from Project 10-33 on the future of airport access... Stay tuned! ✈️🚄
#AirTravel #aviation #mobility #multimodality #STEM #transportation
Are you a MaaS innovator? 📊 Trufi Association is your indispensable partner for GTFS and integration into apps, analytics, and transport tech solutions. Join us in revolutionizing transport for the global South and North!
#informaltransport #opendata #opensource #publictransport #sustainabletransport #transportation #ict4d #ict4dev #MaaS #MobilityAsAService #publictransit #mapping #Multimodality #mobility #populartransport #OpenStreetMap #GTFS
We are pleased to welcome Aishik Mandal, who has just started his PhD at the Ubiquitous Knowledge Processing Lab! 👋 Aishik’s research concerns #Privacy-aware #Multimodality and #DialogueSystems. You can find out more about him here: https://jitaishik.github.io/
Wrangling transportation data in the global South is no easy feat. It's our specialty. We harness community crowdsourcing and OpenStreetMap to provide accurate GTFS informed by the people who know the city better than anyone.
#bus #digital #digitaldevelopment #ict4d #ict4dev #informaltransport #MaaS #mapping #Mobility #multimodal #Multimodality #opendata #opensource #opensteetmap #publictransit #publictransport #PublicTransportation #sustainabletransport
Open Source. Open Data. Better Cities. Help us shape urban mobility with one-time or a recurring donation to Trufi Association
#digitaldevelopment #ict4d #ict4dev #informaltransport #innovation #MaaS #mapping #Mobility #morningcommute #multimodal #Multimodality #opendata #opensource #opensourcesoftware #opensteetmap #OpenTripPlanner #publictransit #publictransport #PublicTransportation #software #sustainabletransport #tech #technology #transportation #donate
If you're interested in #multimodality and facial expression pragmatics come to our talk at #ICLC16 today in session 4 (3F), 11.45.
I'll present work from @facesyntax on how facial expressions can modulate the perceived confidence and doubt of spoken answers in English & Chinese.
Come “Share” With Us Our Enthusiasm… On Twitter!
Virginia Calabria & Sophia Fiedler
The International Society for Conversation Analysis, aka ISCA or @ISCAupdates, is an inclusive community with a global outreach.
Wherever people study #everythingILEMCA, they are part of the ISCA community. ISCA organises
https://www.conversationanalysis.org/come-share-with-us-our-enthusiasm-on-twitter/
#Uncategorized #academictwitter #EMCA #everythingILEMCA #ILEMCA #ISCA #LSI #multimodality
Excited to share: after my current postdoc at the FACESYNTAX project finishes, I'll join @ozyurek_a at the Multimodal Language Department @mpi_nl next year!
We'll use #VR and mocap to probe multimodal communication and language evolution in unprecedented ways. Looking forward to pushing boundaries with wild, insightful experiments. 🕹️😎
#Multimodality #VR #Interaction #LanguageEvolution
RT @JoDoppelbauer: Honoured to participate today at @STARS4Rail webinar.
Uptaking advanced technologies by SMEs in the #Railway sector can only boost its strategic importance within the #multimodality framework. https://t.co/Ulp97RWO4H
🐦🔗: https://n.respublicae.eu/ERA_railways/status/1664644568589934593
#Design #Outlooks
Decoding the Future: the evolution of intelligent interfaces · How we interact with technology is about to change significantly https://ilo.im/134u0j
_____
#IntelligentInterface #ProductDesign #UxDesign #IxDesign #UiDesign #DigitalDesign #WebDesign #AI #AR #VR #Ubiquity #ContextAwareness #Multimodality #Collaboration
EU-Initiative für besseren Vergleich von Verkehrsmitteln stößt auf Kritik https://www.euractiv.de/section/verkehr/news/eu-initiative-fuer-besseren-vergleich-von-verkehrsmitteln-stoesst-auf-kritik/?utm_source=dlvr.it&utm_medium=mastodon #Buchung #DGMove #Drittanbieter #Multimodalität #multimodality
Critics take aim at airline exemption from draft EU travel law https://www.euractiv.com/section/aviation/news/critics-take-aim-at-airline-exemption-from-draft-eu-travel-law/?utm_source=dlvr.it&utm_medium=mastodon #DGMOVE #multimodality #ticketing
Critics take aim at airline exemption from draft EU travel law https://www.euractiv.com/section/aviation/news/critics-take-aim-at-airline-exemption-from-draft-eu-travel-law/?utm_source=dlvr.it&utm_medium=mastodon #DGMOVE #multimodality #ticketing
Enjoying @LingLass's methodological talk on compiling a corpus of internet memes on health-related topics #ICAME44! #multimodality #CorpusLinguistics
Have a look at my new post talking about Instagram stories!
https://medium.com/@cherrie99jones/show-and-tell-multimodality-and-the-use-of-instagram-stories-563aa8bc2b4c
#instagram #stories #multimodality
Are you curious about multimodality in storytelling?
My latest blog post explores the benefits and challenges of using multiple modes of communication, such as images, text, and sound, to convey a message.
Learn how I used Instagram stories to create an immersive experience and engage a wider audience.
https://medium.com/@s5104976/multimodality-in-storytelling-905ccf9857e4
"Trufi Association collaborated with the Stadtnavi project to develop a native app for iOS and Android... the resulting platform is a testament to the benefits of collaboration and open-source development."
– Axel Thévenet
#development #project #android #ios #collaboration #opensource #sustainability #publictransport #multimodality
Reminder: I offer 3 positions for #postdocs/#PhD students in #emotionanalysis from text (1 w/ #argumentmining, 2 w/ #multimodality/#computervision). Apply until May 10, meet me @eaclmeeting. Details: https://www.romanklinger.de/projects/job-emcona.pdf and https://www.romanklinger.de/projects/job-item.pdf #NLProc #eacl2023
🧠 Greg Brockman talks about the #future of #ChatGPT in a TED talk.
⚠️ Will it be an everything app and the virtual assistant of our dreams? I don't know, but the new features are amazing!
#AI #GPT4 #LLM #ArtificialIntelligence #OpenAI #From #multimodality #TextToImage
Me:
What would be the best hashtags to use for that post?
ChatGPT:
Based on the content of the post, the following hashtags may be appropriate:
#AGI #AI #ArtificialIntelligence #MachineLearning #DeepLearning #DataScience #AlphaZero #AlphaGo #MetaLearning #SelfCompetition #MultiModality #Robotics #PhysicalInteractions #Research
All right, here goes! I’m going to try my hand at a bit of #sociolinguistics #scicomm here and describe my latest publication for a lay audience. It’s my first try at this, so be gentle (but do ask questions if you have any!).
My study helps answer the question “What is identity, exactly, and how does it work?” It’s part of a long tradition of studies that have answered parts of that question, dating back several decades now. /1
#identity #multimodality #positioning #MembershipCategorization
So far I haven't really talked about my #research here, but I've seen other #academics announcing their books, so I think I'll at least try this out and see how it goes?
I'm one of the three editors of the new volume #Multimodal #Communication in #Intercultural #Interaction, and if you're interested in purchasing a copy for yourself (or more likely getting someone at your #university to do so), there's a hefty discount code in this flyer!
Join us on zoom in January for a wonderful line-up of speakers at our workshop “Multimodal Digital Curating” Registration link and full programme👇
Please register here:
https://uni-koeln.zoom.us/meeting/register/tJwqceusrDstHdG68_57CwtvJ32NqzFRI4MH
Full program: https://agmedien.de/multimodal-digital-curating/
#multimodality #digitalcuration #mediaanthropology #collaboration #experimentation
Viime päivien Lintula-sekoilut palauttivat mieleen vuoden 2013, jolloin minulla mielestäni oli kanttia jopa sanoa jotain Twitterin merkityksestä tapauksessa "Tv-viihde sosiaalisen median pyörityksessä: Conan O'Brienin tapaus" Lähikuva-julkaisussa. Toiset ajat, toisen ajan some!
Soundtrack Charles Aznavour: “Yesterday When I Was Young”
#Lähikuva #ConanOBrien #TalkShowHistory #TwitterHistory #FanCultures #Multimodality
I extended the deadline, so that everybody can enjoy their vacation.
#callforpapers #CallForPaper #multimodality #diversity #inclusiveness #literatureeducation #l1education
A quick squib about some data I've been working on recently for our #SWELL project.
https://tim792.wixsite.com/swell/post/using-gesture-and-taste-to-solve-a-vocabulary-problem
#EMCA #Multimodality #Multisensoriality #L2Talk #Multilingualism
How can we assess multimodal constructs❓Great discussion on #multimodality and #languagetesting during the Cyril Weir Lecture by @drjenrowsell@twitter.com #LTF2022 @UKALTA2@twitter.com
Since a lot has happened lately, it could be a good time to re-do an #introduction / #introductions.
As a political sociologist I work with text, digital methods & film on matters of #security, #counterterrorism & #surveillance, from a critical perspective.
I currently run the project @securityvision on the politics of computer vision with amazing media artists/designers/visual anthropologists: Ruben van de Ven (@r), Cyan Bae, Ildikó Plájás & Elka Smith.
At Leiden University I'm trying to promote research through making (#film, #photography, #art & other methods) at @recntr, together with Mark Westmoreland and Julian Ross.
I geek out on photo & cinema #cameras & #lenses as well as #floss and #coding, although more from the outside. Working on my first "hello world" in #python
I'm likely to post on #security #computervision #biometrics #facialrecognition #masssurveillance #privacy #technology #film #cinema #multimodality #academia
Machine Learning: Auf der Suche nach einem Alleskönner-Algorithmus