{'fraction_non_alphanumeric': 0.03783095723014256, 'fraction_numerical': 0.014689409368635439, 'mean_word_length': 4.59717868338558, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning Energy-Based Models With Adversarial Training

We study a new approach to learning energy-based models (EBMs) based on adversarial training (AT). We show that (binary) AT learns a special kind of energy function that models the support of the data distribution, and the learning process is closely related to MCMC-based maximum likelihood learning of EBMs. We further propose improved techniques for generative modeling with AT, and demonstrate that this new approach is capable of generating diverse and realistic images. Aside from having competitive image generation performance to explicit EBMs, the studied approach is stable to train, is well-suited for image translation tasks, and exhibits strong out-of-distribution adversarial robustness. Our results demonstrate the viability of the AT approach to generative modeling, suggesting that AT is a competitive alternative approach to learning EBMs.


Introduction
In unsupervised learning, energy-based models (EBMs) [37] are a class of generative model that uses an energy function to model the probability distribution of the observed data. Unlike explicit density models, EBMs model the unnormalized density function, which makes it difficult to evaluate the likelihood function. Maximum likelihood learning of EBMs hence makes use of the likelihood function's gradient which can be approximated using Monte Carlo methods. Each iteration of the learning process involves first generating synthesized data by sampling from the current model, and then updating the model to maximize the energy difference between synthesized data and observed data. This process leads to an energy function that outputs low energies on the data manifold and high energies on other regions. EBMs find applications in image restoration (denoising, inpainting, etc), out-of-distribution detection, and various sample generation tasks. The main difficulties of training EBMs lie in the computational challenges from the sampling procedure and some training stability issues [12,44,43,16,17,11,71,61].
Another line of work on adversarial training (AT) show that adversarially robust classifiers learn high-level, interpretable features, and can be utilized to solve arXiv:2012.06568v4 [cs.LG] 27 Dec 2022 various computer vision tasks including generation, inpainting, super-resolution, and image-to-image translation [57,28,13,51]. Compared to state-of-the-art generative models, this AT approach does not provide a competitive generation performance and is therefore of limited value in many of these tasks. Nonetheless, the generative properties of the robust classifier suggest that the model has captured the distribution of the training data, although the underlying learning mechanism is not yet well understood.
At a high level, both EBMs training and AT are based on the idea of first using gradient-based optimization to generate samples that reach high activation under the current model, and then optimizing the model to minimize its activation on the generated samples. In addition, both approaches synthesize new samples by performing gradient descent on the trained model. These similarities suggest that there are some connections between these two approaches.
In this work we investigate the mechanism by which AT learns data distributions, and propose improved techniques for generative modeling with AT. We focus on binary AT [69] which does not requires class labels and hence naturally fits the generative modeling task. We first analyze the binary AT objective and the corresponding training algorithm, and show that binary AT learns a special kind of energy function that models the support of the observed data. We then draw a connection between AT and MCMC-base maximum likelihood learning of EBMs by showing that the binary AT objective can be interpreted as a gradientscaled version of the likelihood objective in EBMs training, and the PGD attack can be viewed as an non-convergent sampler of the model distribution. This connection provides us with intuition of how AT learns data distributions from a maximum likelihood learning perspective, and suggests that binary AT can be viewed as an approximate maximum likelihood learning algorithm.
We further propose improved techniques for generative modeling with AT based on the above analysis. Our empirical evaluation shows that this AT approach provides competitive generation performance to explicit EBMs, and at the same time is stable to train (just like regular adversarial training), is wellsuited for image translation tasks, and exhibits strong out-of-distribution adversarial robustness. The main limitation of the studied approach is that it cannot properly learn the underlying density function of the observed data. However, this problem is not unique to the studied approach -most existing work on learning EBMs relies on short-run non-convergent sampler to improve the training efficiency, and the learned model typically does not have a valid steady-state that reflects the distribution of the observed data [44,43].
In summary, the contributions of this paper are: 1) We show that binary AT learns a special kind of energy function that models the support of the data distribution, and the learning process is closely related to MCMC-based maximum likelihood learning of EBMs. 2) We propose improved techniques for generative modeling with AT, and demonstrate competitive image generation performance to state-of-the-art explicit EBMs. 3) We show that the studied approach is stable to train, has competitive training and test time sampling efficiency, and can be applied to denoising, inpainting, image translation, and worst-case out-of-distribution detection.

Related Work
Learning EBMs. Due to the intractability of the normalizing constant in the EBMs likelihood function, maximum likelihood learning of EBMs makes use of the gradient of the log-likelihood which can be approximated using MCMC sampling. Recent work [63,12,44,43] scaling EBMs training to high-dimensional data performs sampling using SGLD [60] and initialize the chain from a noise distribution. The sampling process involves estimating the model's gradient with respect to the current sample at each step and therefore has high computational cost. To improve the sampling efficiency, many authors consider shortrun non-convergent SGLD sampler in combination with a persistent sampling buffer [12,43,16,17,61,11]. Although a short-run sampler is sufficient for learning a generation model, the resulting energy function typically does not have a valid steady-state [44,43]. The mixing time of the sampling procedure also depends on how close the chain-initialization distribution is to the model distribution. A recent trend hence considers initializing the sampling chain from samples produced by a generator fitted on the target distribution [62,35,20,45,21,17,61,67,1,42,68].
Maximum likelihood learning of EBMs also has some training stability issues, and various techniques have been developed to address these issues. These techniques include 1) using weight normalization [50], Swish activation [47], gradient clipping, and weight decay (see [61]), 2) gradient norm clipping on model parameters and using a KL term in the training objective (see [11]), 3) adjusting learning rate and SGLD steps during training and adding Gaussian noise to input images (see [16]), 4) gradient clipping on SGLD and model parameters and spectral normalization (see [12]), and 5) multiscale training and smooth activation functions (see [71]). Overall, there does not seem to have a consensus on how to stabilize EBMs training. Due to the computational challenge of MCMC sampling and stability issues, the successful application of EBMs to modeling high-dimensional data such as 256 × 256 images is only achieved in some very recent works [61,71].
Aside from MCMC-based maximum likelihood learning of EBMs, alternative approaches for learning EBMs exist. Score matching [27] circumvents the difficulty of estimating the partition function by directly modeling the derivatives of the data distribution. Score matching has recently been successfully applied to modeling large natural images and achieves competitive performance to stateof-the-art generative models such as GANs [53,54,55,26]. Noise contrastive estimation (NCE) [19] learns data distributions by contrasting the observed data with data from a known noise distribution. Similar to our approach, NCE makes use of a logistic regression model. The main difference is that in NCE, the logit of the classifier is the difference in log probabilities of the model distribution and the noise distribution, whereas in our approach the logit directly defines the estimator (i.e., the energy function). Unlike other EBMs, NCE typically does not scale well to high-dimensional data [19,5,48].
Maximin interpretation of EBMs. When the noise term in the SGLD sampler is disabled, the learning process of EBMs can be interpreted as solving a maximin game [65, 66,64]. This interpretation coincides with our formulation in Eq. (5). The key differences lie in the value function, the setting of the sampler (SGLD vs. PGD attack), and the Markov chain initiation distribution.
Understanding and Improving AT Generative Model. Our work is related to [73,59] which are also attempts to understand and improve AT's generative capability. [73,59]' focus is on the supervised setting, where they use the concept of learning class-conditional energy functions to understand adversarially robust classifiers' generative capability. Our analysis is in an unsupervised setting, where we attempt to understand binary AT's generative property by connecting it with the standard EBMs formulation (Eq. (1)). Although [59] also considers the unsupervised scenario, their unsupervised generative model is a contrastive learning model [30] which requires training samples to perform testtime sampling. Our generative model is based on binary AT and follows the standard practice of MCMC sampling from the learned energy function. In addition to the theoretical analysis, we propose improve training techniques that allow us to obtain a significantly better FID on CIFAR-10 (Tab. 2) and successfully scale the training to 256 × 256 datasets.

Energy-Based Models
Energy-based models (EBMs) [37] represent probability distributions by converting the outputs of a scalar function f θ into probabilities through a Gibbs distribution: where the normalizing constant Z(θ), also known as the partition function, is an integral over the unnormalized probability of all states: The energy function is defined as E θ (x) = −f θ (x), and thus has the property of attributing low energy outputs on the support of the target data distribution and high energy outputs in other regions. For many interesting models, the partition function Z(θ) is intractable, and therefore maximum likelihood estimation (MLE) of the model parameters θ is not directly applicable. Standard maximum likelihood learning of EBMs makes use of the gradient of the log likelihood function. Denote the distribution of the observed data as p data , the gradient of the log likelihood takes the form Intuitively, maximizing log-likelihood with this gradient causes f θ (x) to increase on p data samples and decrease on samples drawn from p θ ; when p θ matches p data , the gradient cancels out and the training terminates.
where p 0 is some random noise distribution. A proper SGLD sampler requires a large number of update steps in order for the distribution of sampled data to match p θ . Due to the high computational cost of this sampling process, many authors resort to short-run

Binary Adversarial Training
Binary adversarial training [69] is a method for detecting adversarial examples. In a K class classification problem, the detection method consists of K binary classifiers, with the k-th binary classifier trained to distinguish clean data of class k from adversarially perturbed data of other classes. A committee of K binary classifiers then provides a complete solution for detecting adversarially perturbed samples of any classes. Denote the data distribution of class k as p data , the mixture distribution of other classes as p 0 = 1 K−1 i=1,...,K,i̸ =k p i , the k-th binary classifier is trained by maximizing the objective where D : X ⊆ R d → [0, 1] is the classification function, and B(x, ϵ) is a neighborhood of x: B(x, ϵ) = {x ′ ∈ X : ∥x ′ − x∥ 2 ≤ ϵ}. In practice, D is defined by applying a logistic sigmoid function to the output of a neural network: where f θ is a neural network with a single output node and parameters θ. The inner minimization in Eq. (4) is solved using the PGD attack [38,36], a first-order method that employs an iterative update rule of (l 2 -based attack): where λ is some step size, and Proj is the operation of projecting onto the feasible set B(x, ϵ). Because the gradient vector in Eq. (6) is normalized to have unit norm, we can equivalently implement the attack by directly performing gradient ascent on f θ : ).
4 Binary AT Generative Model In this section we develop a generative model based on binary AT. We first analyze the optimal solution to the binary AT problem, and then investigate the mechanism by which binary AT learns the data distribution, and finally interpret the learning process from the maximum likelihood learning perspective.
Our main result is that under a proper configuration of perturbation limit and p 0 data, binary AT learns a special kind of energy function that models the support of p data . Based on these theoretical insights, we proposed improved training techniques.

Optimal Solution to the Binary AT Problem
We consider the optimal solution of Eq. (4) under the scenario of unbounded perturbation: B(x, ϵ) = X . This allows us to further simplify the PGD attack by removing the Proj operator: Perturbing p 0 samples can be thought of as moving p 0 samples via a translation function T (x) = x + ∆ x , with ∆ x being the perturbation computed on sample x. We can write the density function of the perturbed distribution p T using random variable transformation: The inner problem in Eq. (4) can then be interpreted as determining the distribution which has the lowest expected value of log(1 − D(x)): The objective of the outer problem is then the log-likelihood in a logistic regression model which discriminates p data samples from p * T samples: We can equivalently formulate Eq. (4) as a maximin problem and obtain its optimal solution by following the standard approach to solving maximin problems: (4), where D * outputs 1 2 on Supp(p data ) and ≤ 1 2 outside Supp(p data ), and p * T is supported in the contour set {D = 1 2 }.
Proof. See the supplementary materials.
The above maximin problem can also be interpreted as a two-player zero-sum game, and is closely related to GANs [15]'s minimax game which has the form The game-theory point of view provides a convenient way to understand their differences. We include a game theory-based analysis of max D min p T U (D, p T ) and a comparative analysis of GANs in the supplementary materials.

Learning Mechanism
Proposition 1 states that by solving max D min p T U (D, p T ) we can obtain a D that outputs 1 2 on the support of p data and ≤ 1 2 on other regions. This result is obtained by assuming that for any D, the inner minimization Eq. (10) is always perfectly solved. In practice, when D is randomly initialized, it has many local maxima outside the support of p data . Because the inner minimization is solved by taking p 0 samples and then performing gradient ascent on D with Eq. (8), this process can get trapped in different local maxima of D. Hence we can think of this process as searching for these local maxima and then put the perturbed p 0 data in these regions. Then in the model update stage (outer maximization), D is updated by increasing its outputs on p data samples and decreasing its outputs on the perturbed p 0 data. By repeating this process, local maxima get suppressed and the model learns to correctly model Supp(p data ).
The algorithm for solving the maximin problem is described in Algorithm 1. Fig. 1 left panel shows the 2D simulation result of the algorithm when the p 0 dataset contains random samples from the uniform distribution. It can be seen that when the algorithm converges, local maxima outside Supp(p data ) are suppressed, and D (approximately) outputs 1 2 on Supp(p data ) as predicted by Proposition 1. Meanwhile, D retains the gradient information for translating out-distribution samples to Supp(p data ).
Because the PGD attack is deterministic gradient ascent, its ability to discover different local maxima depends on the diversity of p 0 samples. Fig. 1 right panel shows that when p 0 data is concentrated in the bottom left corner, the final D still has local maxima outside the support of p data . These local maxima are not suppressed because they were never discovered by the perturbed p 0 data.
The above analysis reveals how binary AT learns data distributions: the learning starts with a randomly-initialized D solution, and then iteratively refine the solution by suppressing local maxima outside the support of the target distribution. This process is similar to EBMs training where the model distribution's spurious modes are constantly discovered by MCMC sampling and subsequently suppressed in the model update stage. However, unlike the EBMs likelihood objective Eq. (2), the AT objective Eq. (5) cannot properly learn the density function, but can only capture its support. This is corroborated by the 2D experiment where D outputs 1 2 uniformly on the support of p data (blue points).
Algorithm 1 Binary Adversarial Training

Maximum Likelihood Learning Interpretation
We next consider the learning process of binary AT from a maximum likelihood learning point of view. Both binary AT and MCMC-based EBMs learning employ an iterative optimization algorithm, where in each iteration the contrastive data is computed by performing gradient ascent on the current model, and then the model is updated by maximizing its outputs on the observed data and minimizing its outputs on the contrastive data. The following analysis shows that the PGD attack can be viewed as a non-convergent sampler of the model distribution, and the binary AT objective Eq. (11) can be interpreted as a gradient-scaled version of the EBMs objective Eq. (2). Tab. 1 summaries their key differences. Contrastive Data Computation. In EBMs training, the contrastive data is computed by MCMC-sampling, typically with Langevin dynamics Eq. (3). In binary AT, the contrastive data is computed using the PGD attack Eq. (8). Table 1. Key differences between binary AT and maximum likelihood EBMs.

Objective gradient
EBMs: EBMs: A noise distribution or a distribution close to pdata Binary AT: A real and diverse out-distribution dataset (80 million tiny images for CIFAR-10 and ImageNet for 256x256 datasets).
Comparing Eq. (8) with Eq. (3), we find that both approaches compute the contrastive data by first initializing from some out-distribution data, and then performing gradient ascent on f θ . The main differences are that the PGD attack does not have the noise term, and makes use of normalized gradient. Intuitively, the noise term enables the sampler to explore different modes by helping gradient ascent escape local maxima. Although the PGD attack does not have the noise term, its ability to explore different modes can be enhanced by using a diverse p 0 dataset (Fig. 3).
In the PGD attack, as the normalized gradient vector has unit norm, the perturbation imposed on x i is λ; in a K iterations of the update, the overall Hence with the PGD attack we can more easily control the distribution of the contrastive data. In contrast, Langevin dynamics adjusts x i in a scale that corresponds to the magnitude of the gradient of f θ at x i ; when f θ is updated during training, the overall perturbation may undergo a large change. This behavior of Langevin dynamics can be a source of some training stability issues [43].
Gradient of the Training Objective. By definition Eq. (5), the gradient of D's training objective Eq. (11) takes the form (14) Comparing the above equation with Eq. (2) we find both equations consisting of gradient terms that yield similar effects: the first term causes f θ outputs on p data samples to increase, and the second causes f θ outputs on the contrastive samples to decrease. Specifically, as (1 − σ(f θ (x))) and σ(f θ (x)) are scalars in the range 0 to 1, the two gradient terms in Eq. (14) are respectively the scaled versions of the gradient terms in Eq. (2). It should be noted that although these scalars do not change the gradient update direction of individual terms in the model parameter space, the overall gradient update directions of Eq. (14) and Eq. (2) can be different.
Eq. (14) also helps to understand why binary AT can only learn the support of the observed data. In Eq. (2), when p θ (x) matches p data , the gradient cancels out and training terminates, whereas in Eq. (14), when p * T matches p data the gradient becomes E x∼p data [(1 − 2σ(f θ (x)))∇ θ f θ (x)] and only vanishes when σ(f θ (x)) = 1 everywhere on the support of p data . This result is consistent with Proposition 1 and the 2D experiment result.

Improved Training of Binary AT Generative Model
Diverse p 0 Data. As discussed in Sec. 4.2, a diverse p 0 dataset improve the PGD attack's ability to explore different local maxima of D. To validate this 2D intuition generalizes to high dimensions, we evaluate the image generation performances of models trained with different p 0 datasets. Fig. 3 shows that the best FID is obtained when p 0 is the most diverse dataset among the considered p 0 datasets. (Note that we have used the same setting of the PGD attack and source images dataset to perform test-time generation in these three experiments. ) We follow existing work on adversarial training and use a p 0 dataset that contains real data samples to train the model. Using a real dataset (as opposed to a noise distribution) helps the model achieve out-of-distribution adversarial robustness (Sec. 5.2 OOD detection) and learn informative gradient for transforming real out-distribution samples (not just noise samples) into valid samples of p data . The latter can be a useful feature in image translation applications (Sec. 5.2). The setting of p 0 in our experiments can be found in Tab. 1.
Training With Unconstrained Perturbations. Existing work on using adversarial training to train robust classifiers uses a small, fixed perturbation limit [38]. In the generative modeling task, we would like the perturbed p 0 data to travel in a larger space to find more local maxima. This can be achieved by taking more PGD attack steps (K) in step 3 of Algorithm 1. Fig. 4 shows that a larger K results in better FID scores.
The downside of a large K is that it converges slower because more gradient steps are taken in each iteration (Fig. 5 K = 25 vs. K = 5). To improve the training efficiency we propose a mixed scenario in which we progressively increase the K value during training. We observe that this progressive training scenario converges faster than training with fixed-K ( Algorithm 2 Progressive Binary Adversarial Training for number of training iterations do 3:  Regularization. While other generative models typically require some forms of regularization, the proposed model can be trained successfully without using any regularization. One trick that we find beneficial for achieving better FID (Fig. 2, R 1 reg = 30, pretrained vs. from scratch) is to pretrain the D model on the ImageNet classification task. (This requires adding auxiliary output nodes which are ignored when later training the D model.) When using the pretrained model, we find it necessary to use R 1 regularization [40], otherwise the FID stops improving after a few hundred iterations (Fig. 2, R 1 reg = 0 pretrained). Note that when D is trained from scratch, R 1 regularization is not strictly required, but adding the regularizer does not hurt the performance (Fig. 2, R 1 reg = 0 from scratch vs. R 1 reg = 30 from scratch).

Experiments
In this section we provide an empirical evaluation of the proposed AT generative model. We first evaluate the approach's image generation performance and then demonstrate its applications to image-to-image translation and worst-case out-of-distribution detection. We further provide an analysis of the proposed approach's training stability in Sec. 5.3.
In the supplementary materials we provide experiment setup details including model architectures, training hyperparameters, sample generation settings, and evaluation protocols. We also include additional results including sampling efficiency analysis, uncurated image generation and image translation results, and demonstration of applications to denoising, inpainting, and compositional visual generation [10]. The interpolation results and nearest neighbor analysis in the supplementary materials suggest that our model captures the manifold structure of the observed data, as opposed to simply memorizes the data samples.

Image Generation
Tab. 2 shows that on CIFAR-10 [34] our approach achieves the best Inception Score (IS) [49] and FID [25] among AT generative models. Our approach also improves over state-of-the-art explicit EBMs in terms of IS, and at the same time has a slightly worse FID. Compared to VAEBM [61], our method does not require an auxiliary model to train, and has better test time sampling efficiency (see supplementary materials). Diffusion Recovery [14] trains a sequence of conditional EBMs, with each one defining the conditional distribution of a noisy sample given the same sample at a higher noise level. Similar to score-based approaches, these conditional EBMs do not directly model the data distribution of the observed data, so it is unclear how these models can be applied to tasks which require explicit knowledge of the data distribution (e.g., OOD detection).
Tab. 3 shows that on CelebA-HQ 256 [31] our method outperforms or is on par with state-of-the-art generative models except GANs. On LSUN Church [70] our method outperforms a latest energy-based model VAEBM [61] (the authors only provided the 64 × 64 result), but falls bellow DDPM and GANs. Fig. 6 shows sample image generation results.

Applications
Image-To-Image Translation. Fig. 7 shows that the AFHQ-CAT model can be used to transform CelebA-HQ images into cat images, and vice-versa. Note that these two models are trained independently without knowledge of the source domain, indicating that our approach may generalize better to unseen data than approaches (e.g., pix2pix [29], CycleGAN [72], and StarGAN [6]) that explicitly use the source domain dataset to train the model. The translation results may be further improved by finetuning the trained model on the source domain dataset, or including the source domain data in the p 0 dataset during training. The proposed approach is also more flexible than approaches that employs a fixed  [2]. OE, RATIO, and JEM all perform OOD detection by utilizing a classifier that has low confidence predictions on the out-distribution data (clean or worst-case). In RATIO, the worstcase out-distribution data is computed by performing the PGD attack on 80 million tiny images, whereas in JEM it is computed via Langevin dynamics initialized from uniform random noise. RATIO and our method's strong outdistribution adversarial robustness demonstrates the benefit of using a real and diverse out-distribution dataset to train the model. Our method does not make use of class labels and therefore can be considered as a binary variant of RA-TIO. On CelebA-HQ 256, AFHQ-CAT, and LSUN-Church our model similarly achieves strong out-distribution adversarial robustness (supplementary materials). These results suggest that our generative model can be applied to detect both naturally occurring OOD data and adversarially created malicious content.

Training Stability Analysis
To gain some insight into the training stability of our approach we investigate whether the PGD attack can be used with the EBMs training objective Eq. (2). Specifically, in Algorithm 2, we perform step 5's update on θ using the gradi- . We observe that even under a small learning rate of 1e − 6, 1 i ) quickly increases and eventually overflows. This suggests that the stability of the AT approach can be largely attributed to the log-likelihood objective Eq. (11). We argue that the stability is due to the gradient cancelling effect of this objective: when f θ has a large positive output on a sample x ∼ p data , 1 − σ(f θ (x)) approaches 0 and therefore the corresponding scaled gradient in Eq. (14) vanishes, and similarly σ(f θ (x * ))∇ θ f θ (x * ) vanishes when f θ has a large negative output on a sample x * ∼ p * T . In contrast, the EBMs objective Eq. (2) does not have constraints on f θ 's outputs and is therefore prone to divergence. Table 4. CIFAR-10 standard and worst-case OOD detection results (AUC scores). We use the same settings of AutoAttack [7], number of OOD samples, and perturbation limit as in [2]

Conclusion
We have studied an AT-based approach to learning EBMs. Our analysis shows that binary AT learns a special kind of energy function that models the support of the observed data, and the training procedure can be viewed as an approximate maximum likelihood learning algorithm. We proposed improved techniques for generative modeling with AT, and demonstrated that the proposed method provides competitive generation performance to explicit EBMs, has competitive sampling efficiency, is stable to train, and is well-suited for image translation tasks. The proposed approach's strong out-distribution adversarial robustness suggests its potential application to detecting abnormal inputs and/or adversarially created fake content. In this section we provide a comparative analysis of the proposed AT generative model and GANs [15]. The proposed approach learns data distribution by solving the maximin problem while GANs learn a generator function G by solving the minimax problem The generator G implicitly defines a distribution p g by mapping a prior distribution p z from a low-dimensional latent space Z ⊆ R z to the high-dimensional data space X ⊆ R d . Plugging p g into Eq. (6), we get: Comparing Eq. (5) with Eq. (7) we find both problems making use of the standard log-likelihood objective for binary classification, but have a reversed order of minimization and maximization. In fact, both formulations solve a twoplayer zero-sum game, a mathematical representation of a situation in which one player's gain is balanced by another player's loss. This game can be described by the payoff function f : R p+q → R, which represents the amount of payment that one player (player 1) makes to the other player (player 2). The goal of player 1 is to choose a strategy u ∈ R p such that the payoff is minimized, while the goal of player 2 is to choose a strategy u ∈ R q such that the payoff is maximized. Depending on the order of maximization and minimization, the best strategies for both players, and the optimal payoff, can be solved via min u max v f (u, v) or max v min u f (u, v). In Eq. (5), U (D, p T ) is the payoff function, and the goal of player p T is to choose a strategy p * T such that the payoff is minimized, whereas the goal of player D is to choose a strategy D * such that the payoff is maximized. This maximin game is played by following such a rule: player D makes the first move by choosing a D; player p T , after learning that player D has made the move, will choose a p T to minimize its payment, which results in a payoff of min p T U (D, p T ); player D, who is informed of player p T 's strategy, will chooses a D such that the worse case payoff min p T U (D, p T ) is maximized, which results in an overall payoff of max D min p T U (D, p T ). The best strategies of both players and the maximum payoff can be derive from Proposition 1 : In the maximin game max D min p T U (D, p T ), the best strategy for player D is to choose a D * that outputs 1 2 on Supp(p data ) and ≤ 1 2 outside Supp(p data ), the best strategy for player p T is to choose a p * T which is supported in {x : D(x) = 1 2 }, and the maximum payoff is − log(4).
In Eq. (7), U (D, p g ) is the payoff function. Similar to Eq. (5), the goal of player p g is to minimize the payoff, and the goal of player D is to maximize the payoff. In contrast to Eq. (5), player p g makes the first move. The solution to this minimax game is analyzed in [15]: the best strategy of player p g is to choose a p * g which minimizes the Jensen-Shannon divergence (JSD) between p g and p data : p * g = arg min pg JSD(p g ∥ p data ) = p data , and the best strategy of player D is to choose D * (x) = Under these strategies, the payoff function U measures the JSD between p g and p data : U (D * , p * g ) = − log(4) + 2 · JSD(p * g ∥ p data ) = − log(4), which coincides with the U solution in the maximin game. Note that in the minimax game, D * does not need to be defined outside Supp(p g ) ∪ Supp(p data ) [15].
The optimal solutions to these two formulations are summarized in Tab. 1. The pseudo code for solving the minimax problem is outlined in Algorithm 3. Fig. 1 shows the simulation results in two settings where p 0 data is respectively uniformly distributed (left panel) and concentrated in the lower left corner (right panel). In can be seen that in both cases p * T matches p data when the algorithm converges. The right panel shows that when p 0 data is concentrated in the lower left corner, the D solution has undefined outputs outside Supp(p data ).
We find these two formulations giving rise to different applications. The minimax formulation is ideal for learning a generator model that can produce a distribution that matches p data . The discriminator, because of its undefined behavior outside Supp(p data ), may not be very useful for some downstream tasks such as out-of-distribution detection. In the maximin formulation, as we have discussed in the main text, can be used for sample generation, image-to-image translation, image restoration such as denoising and inpainting, and (worst-case) out-of-distribution detection. Table 1. Optimal solutions to the minimax problem and maximin problem Minimax (GANs) Eq. (7) Maximin (ours) Eq. (5) Algorithm 3 Solving the minimax problem  Training. We use Algorithm 2 to train the models. The training hyperparameters for each task can be found in Tab. 3. For the 256 × 256 tasks, we pretrain the D model on the ImageNet classification task. To mitigate overfitting, we perform random resized cropping, random horizontal flipping on p data samples. The performance (FID score) of the model is monitored during training and the best-performing model to used to report the final FID score. The CIFAR-10 worst-case OOD detection model is trained using in-and outdistribution adversarial training [2], where in-distribution AT uses a l 2 -ball of radius 0.25 and PGD attacks of steps 10 and step-size 0.1, and out-distribution AT uses a l 2 -ball of radius 0.5 and PGD attacks of steps 10 and step-size 0.1. Following [2], we use a batch size of 128 and use the recommended AutoAugment policy from [8]. The model is trained for 400 epochs using a SGD optimizer with a fixed learning rate of 0.1. Sample Generation. The generated samples for FID and IS evaluation are produced by performing PGD attacks on 50K samples randomly drawn from the p 0 dataset. The settings for the p 0 dataset and the PGD attack can be found in Table 4.

Training and Test Time Sampling Efficiency
Tab. 5 shows that our method has competitive training and test time sampling efficiency to state-of-the-art EBMs. Although VAEBM typically requires much fewer update steps than our method, its per-step efficiency is much worse (Tab. 6), suggesting that its VAE component has considerable computational complexity. We also observe that the quality of our generated samples is not sensitive to the number of sampling steps as long as the overall perturbation (#step × step-size) remains the same (Tab. 7). This allows us to use a much larger step size than the one used during training to speedup test time sampling in real applications.

Extend Results on Worst-Case Out-Of-Distribution Detection
Tab. 8 shows that under a PGD adversary with l 2 radius 7.0 our model exhibits strong out-distribution robustness. (Note that according to [2], a perturbation of 7.0 is already large enough to make undefended models (e.g., OE [24]) fail completely at the OOD detection task).

Extended Results on Generation
Additional results are summarized below: -Uncurated generation samples. Fig. 2, Fig. 4, Fig. 6, and Fig. 7 show the uncurated generated samples on CIFAR-10, CelebA-HQ 256, AFHQ-CAT 256, and LSUN-Church 256. Note that we have used the same seed images (Fig. 21) to generated these results. We find that some generated images contain artifacts. By first applying Gaussian smoothing (σ = 10) to the source images (p 0 data), we are able to obtain more visually pleasing results (Fig. 5). The generated samples contain less artifacts, but have a slightly worse FID. The smoothing filters out high frequency components, and seems to be playing a similar role as reduced-temperature sampling [58,61] and the "truncation trick" [4], where better-looking results (typically with reduced diversity) can be generated from latent noise sampled from the high density area of the latent space. -Nearest Neighbor Analysis. Fig. 3, Fig. 8, Fig. 9, and Fig. 10 show the pixel space and inception feature space nearest neighbors of the generated samples on CIFAR-10, CelebA-HQ 256, AFHQ-CAT 256, and LSUN-Church 256. Note that none of the nearest neighbors resemble the generated samples, suggesting that the models have not memorized the training data. -Interpolation. Fig. 11, Fig. 12, and Fig. 13 show the interpolation results on CelebA-HQ 256, AFHQ-CAT 256, and LSUN-Church 256. The interpolation works reasonable well even on AFHQ-CAT where only about 5000 training images are available. -Intermediate Generation Results. Fig. 14, Fig. 15, and Fig. 16 show the intermediate generation results. It can be seen that the model is capable of transforming natural images into valid images of the target data distribution. In addition, when the number of PGD attack steps is too large, the generated samples become saturated, which suggests that the model, like many EBMs trained with short-run MCMC, do not have a valid steady-state that reflects the distribution of target data. -Compositional Visual Generation. Fig. 17 shows that our model can be composed like regular EBMs [10]. -Denosing and Inpainting. Fig. 18 and Fig. 19 show uncurated denoising and inpainting results on CelebA-HQ 256 and AFHQ-CAT 256. -Image Translation. Fig. 20 shows uncurated image translation results on CelebA-HQ 256 and AFHQ-CAT 256.
Source images Generated images               Original images (1st row), images with additive Gaussian noise of standard deviation of 0.1 (2nd row), and recovered images (last row).