For best experience please turn on javascript and use a modern browser!
You are using a browser that is no longer supported by Microsoft. Please upgrade your browser. The site may not present itself correctly if you continue browsing.
Master AI alumnus Piyush Bagad is the recipient of the KHMW Thesis Prize for Informatica and Informatiekunde. During an award ceremony organized by the Koninklijke Hollandsche Maatschappij der Wetenschappen on 27 November, he will receive a prize of 3000 euros for his thesis entitled “It’s About Time: Time Awareness in Video Foundation Models”. For his master’s thesis, Piyush focused on foundation models. These are a revolutionary class of new AI systems that display emergent behavior and provide impressive abilities to generate text, images, computer programs and more, by learning from huge amounts of data. Famous examples are ChatGPT, Bard and Stable Diffusion. However, the behavior of these foundation models is poorly understood and often unpredictable. Piyush studied video foundation models and questions their temporal awareness.

Time-awareness

The first part of the thesis probes video-language models for their ability to distinguish simple ‘before’ and ‘after’ relationships, using synthetic data, to establish that out of six modern foundation models none passes a simple test of time. Further, a data-efficient post-pretraining procedure is proposed that can inject a sense of time into one such model, without the need to train it again from scratch. While doing so making huge computational savings. The updated model performs well in general temporal video-language reasoning tasks. Demonstrating for the first time that existing video foundation models can be ‘updated’ to inject new behavior. The second, more explorative, study extends temporal awareness to audio-visual models, covering the definition, representation, and benefits of being time-aware on state-changing actions like ‘pouring water’, and reversible actions like ‘turning on/off tap’. It again demonstrates existing foundation models are unable to pass a simple audiovisual time-aware arrow-of-time prediction task.

CVPR conference

Part of Piyush’s thesis has been published as a paper at Computer Vision and Pattern Recognition (CVPR), the world’s most prestigious AI conference. According to Google Scholar, CVPR currently ranks as the fourth most cited scientific publication, just after Nature, The New England Journal of Medicine and Science. Piyush was the first UvA Master AI student to publish at CVPR. 

Amsterdam Merit Scholarship

Piyush was previously a recipient of the Amsterdam Merit Scholarship which supports international students studying in the Master’s programmes at the University of Amsterdam. He was also selected as an ELLIS Honors Student to pursue his Master’s thesis advised by Prof. Cees Snoek in collaboration with Dr. Makarand Tapaswi (IIIT Hyderabad, India) and Prof. Andrew Zisserman at the University of Oxford. Piyush is currently a DPhil student at the University of Oxford.