Nvidia AI plays Minecraft, wins machine-learning conference award.
A paper describing MineDojo, Nvidia’s generalist AI agent that can perform actions from written prompts in Minecraft, won an Outstanding Datasets and Benchmarks Paper Award at the 2022 NeurIPS (Neural Information Processing Systems) conference, Nvidia revealed on Monday.
To train the MineDojo framework to play Minecraft, researchers fed it 730,000 Minecraft YouTube videos (with more than 2.2 billion words transcribed), 7,000 scraped webpages from the Minecraft wiki, and 340,000 Reddit posts and 6.6 million Reddit comments describing Minecraft gameplay.
To stay up to date with latest top stories, make sure to subscribe to this YouTube channel by clicking the button above this video!
From this data, the researchers created a custom transformer model called MineCLIP that associates video clips with specific in-game Minecraft activities.
As a result, someone can tell a MineDojo agent what to do in the game using high-level natural language, such as “find a desert pyramid” or “build a nether portal and enter it,” and MineDojo will execute the series of steps necessary to make it happen in the game.
Foreign [Music] Plays Minecraft wins machine learning Conference award a paper describing mine Dojo nvidia's generalist AI agent that Can perform actions from written prompts In Minecraft when an outstanding data Sets and benchmarks paper award at the 2022 nureps neural information Processing systems conference Nvidia Revealed on Monday to train the mind Dojo framework to play Minecraft Researchers fed it 730 000 Minecraft YouTube videos with more than 2.2 Billion words transcribed seven thousand Scraped web pages from the Minecraft Wiki and 340 000 Reddit posts and 6.6 Million Reddit comments describing Minecraft gamma play to stay up to date With latest top stories make sure to Subscribe to this YouTube channel by Clicking the button above this video From this data the researchers created a Custom Transformer model called Miniclip That Associates video clips with Specific in-game Minecraft activities as A result someone can tell a mind Dojo Agent what to do in the game using High-level natural language such as find A desert pyramid or build another portal And enter it and mindojo will execute The series of steps necessary to make it Happen in the game examples of tasks That mind Dojo can perform mind Dojo
Aims to create a flexible agent that can Generalize learned actions and apply Them to different behaviors in the game As Nvidia writes while researchers have Long trained autonomous AI agents in Video game environments such as Starcraft DOTA and go these agents are Usually specialists in only a few tasks So Nvidia researchers turn to Minecraft The world's most popular game to develop A scalable training framework for a Generalist agent one that can Successfully execute a wide variety of Open-ended tasks the award-winning paper Minidoho building open-ended embodied Agents with internet scale knowledge Debuted in June its authors include Linksy fan of Nvidia and guanzi Wang Yunfen Jian A J mamliker young Kong Yang Heiwaju Andrew Tang Dion Juan uq anonyma Nantumer of various academic Institutions you can see examples of Mind dojo in action on its official Website and the code for mine dojo and Miniclip is available on GitHub [Music]
