Welcome to Replicate’s weekly bulletin! Each week, we’ll bring you updates on the latest open-source AI models, tools, and research. People are making cool stuff and we want to share it with you. Without further ado, here’s our hacker-in-residence deepfates with an unfiltered take on the week in AI.
Editor’s note
This week I’m thinking about how multimedia AI models will lead to real-time interactive world generation, and how it’s the bull case for VR and the metaverse. I talked with fellow Replicant Mattt about it, and watched his talk (see Research Radar below), and I can’t get it out of my mind. (Editor’s note: Neither Mattt nor Replicate are responsible for the following conjectures)
Just this week: You can now fine-tune FLUX.1, Tavus launched their Conversational Video Interface, a “digital twin” API that can looks like a person and does real-time video chat. Puppet-Master adds drag tokens to Stable Video Diffusion so you can close the door on a picture of a microwave. Sketch2scene is a big Rube Goldberg-like project involving several models to get from a crude drawing to a fully playable game world, but it works.
Image generators based on FLUX.1 can do hands and text and all the stuff we previously used to distinguish AI images. It’s the worst this technology will ever be. And it shows the clear desire for the ability to generate worlds.
