Masked Autoregressive Flow for Density Estimation

Masked Autoregressive Flow for Density Estimation

George Papamakarios, Theo Pavlakou, Iain Murray

Intro

There are mainly two neural density estimators:

1. autoregressive model: decompose the joint density as a product of conditionals, and model each conditional in turn
2. normalizing flow: transform a base density into the target density by an invertible transformation with tractable Jacobian

autoregressive model 可以当作是从random external source，通过一个differentiable transfomration而产生数据的一个方式。对于某些autoregressive model这种transformation又是invertible，因此可以当作是一个normalizing flow。

Masked Autoregressive Flow

Autoregressive models as normalizing flows

$$p(xi| x{1:i-1} = \mathcal{N}(f{\mu_i}(x{1:i-1}), \exp(f{\alpha_i}(x{1:i-1}))^2 )$$

To generate the data

1. sample $$u_i \sim \mathcal{N}(0, 1)$$
2. generate $$xi = u_i \cdot \exp f{\alphai}(x{1:i-1}) + f{\mu_i}(x{1:i-1})$$