Replicate a Transformer. Attention is all you need!

Attention Is All You Need

We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We then discuss how this model can be trained on a large and diverse dataset from multiple dexterous robot platforms, including single-arm robots, dual-arm robots, and mobile manipulators. We evaluate our model in terms of its ability to perform tasks via direct prompting, follow language instructions from people and from a high-level VLM policy, and its ability to acquire new skills via fine-tuning.

pi0 : Our First Generalist Policy

We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics. Our best results are obtained by training on a weighted variational bound designed according to a novel connection between diffusion probabilistic models and denoising score matching with Langevin dynamics, and our models naturally admit a progressive lossy decompression scheme that can be interpreted as a generalization of autoregressive decoding.

Denoising Diffusion Probabilistic Models (Part 1)

In this part, we try to code a diffusion model ourselves.

Denoising Diffusion Probabilistic Models (Part 2)

We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples—which subsumes existing diffusion paths as specific instances. Interestingly, we find that employing FM with diffusion paths results in a more robust and stable alternative for training diffusion models. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths. An instance of particular interest is using Optimal Transport (OT) displacement interpolation to define the conditional probability paths. These paths are more efficient than diffusion paths, provide faster training and sampling, and result in better generalization. Training CNFs using Flow Matching on ImageNet leads to consistently better performance than alternative diffusion-based methods in terms of both likelihood and sample quality, and allows fast and reliable sample generation using off-the-shelf numerical ODE solvers.

Flow Matching for Generative Modeling (Part 1)

We try to code a Flow Matching,  but not finished yet.

Flow Matching for Generative Modeling (Part 2)

Database System Notebook

Research

Course

首页

文档整理

时间线

Github

友情链接

关于我

友链

自我介绍

Database System

大二下

Paper

Flow Matching

Transformer

Diffusion Model

author

publish date

featured image

read time

language

audience

external links

email notifications

comments enabled

password

icon

date

type

slug

status

title

summary

Table

Config

Post Board

Post Gallery

类型为Notice的文章将被显示为公告

公告

This Is Where We Start

📔Database System Notebook