diff options
Diffstat (limited to 'report/paper.md')
| -rw-r--r-- | report/paper.md | 22 |
1 files changed, 11 insertions, 11 deletions
diff --git a/report/paper.md b/report/paper.md index 1989472..364e6a5 100644 --- a/report/paper.md +++ b/report/paper.md @@ -19,7 +19,7 @@ Training a shallow GAN with no convolutional layers poses problems such as mode \end{figure} -Mode collapse is achieved with our naive *vanilla GAN* (Appendix-\ref{fig:vanilla_gan}) implementation after 200,000 epochs. The generated images observed during a mode collapse can be seen on figure \ref{fig:mode_collapse}. The output of the generator only represents few of the labels originally fed. When mode collapse is reached loss function of the generator stops improving as shown in figure \ref{fig:vanilla_loss}. We observe, the discriminator loss tends to zero as the discriminator learns to assume and classify the fake 1's, while the generator is stuck producing 1 and hence not able to improve. +Mode collapse is achieved with our naive *vanilla GAN* (Appendix-\ref{fig:vanilla_gan}) implementation after 200,000 batches. The generated images observed during a mode collapse can be seen on figure \ref{fig:mode_collapse}. The output of the generator only represents few of the labels originally fed. When mode collapse is reached loss function of the generator stops improving as shown in figure \ref{fig:vanilla_loss}. We observe, the discriminator loss tends to zero as the discriminator learns to assume and classify the fake 1's, while the generator is stuck producing 1 and hence not able to improve. A significant improvement to this vanilla architecture is Deep Convolutional Generative Adversarial Networks (DCGAN). @@ -54,7 +54,7 @@ We evaluate three different GAN architectures, varying the size of convolutional \begin{figure} \begin{center} -\includegraphics[width=24em]{fig/med_dcgan_ex.pdf} +\includegraphics[width=24em]{fig/med_dcgan_ex.png} \includegraphics[width=24em]{fig/med_dcgan.png} \caption{Medium DCGAN} \label{fig:dcmed} @@ -62,9 +62,9 @@ We evaluate three different GAN architectures, varying the size of convolutional \end{figure} We observed that the deep architectures result in a more easily achievable equilibria of G-D losses. -Our medium depth DCGAN achieves very good performance, balancing both binary cross entropy losses at approximately 0.9 after 5.000 epochs, reaching equilibrium quicker and with less oscillation that the Deepest DCGAN tested. +Our medium depth DCGAN achieves very good performance, balancing both binary cross entropy losses at approximately 0.9 after 5.000 batches, reaching equilibrium quicker and with less oscillation that the Deepest DCGAN tested. -As DCGAN is trained with no labels, the generator primary objective is to output images that fool the discriminator, but does not intrinsically separate the classes form one another. Therefore we sometimes observe oddly shape fused digits which may temporarily full be labeled real by the discriminator. This issue is solved by training the network for more epochs or introducing a deeper architecture, as it can be deducted from a qualitative comparison +As DCGAN is trained with no labels, the generator primary objective is to output images that fool the discriminator, but does not intrinsically separate the classes form one another. Therefore we sometimes observe oddly shape fused digits which may temporarily full be labeled real by the discriminator. This issue is solved by training the network for more batches or introducing a deeper architecture, as it can be deducted from a qualitative comparison between figures \ref{fig:dcmed}, \ref{fig:dcshort} and \ref{fig:dclong}. Applying Virtual Batch Normalization our Medium DCGAN does not provide observable changes in G-D balancing, but reduces within-batch correlation. Although it is difficult to qualitatively assess the improvements, figure \ref{fig:vbn_dc} shows results of the introduction of this technique. @@ -78,7 +78,7 @@ Applying Virtual Batch Normalization our Medium DCGAN does not provide observabl \end{figure} We evaluated the effect of different dropout rates (results in appendix figures \ref{fig:dcdrop1_1}, \ref{fig:dcdrop1_2}, \ref{fig:dcdrop2_1}, \ref{fig:dcdrop2_2}) and concluded that the optimisation -of the droupout hyper-parameter is essential for maximising performance. A high dropout rate results in DCGAN producing only artifacts that do not match any specific class due to the generator performing better than the discriminator. Conversely a low dropout rate leads to an initial stabilisation of G-D losses, but ultimately results in instability under the form of oscillation when training for a large number of epochs. +of the droupout hyper-parameter is essential for maximising performance. A high dropout rate results in DCGAN producing only artifacts that do not match any specific class due to the generator performing better than the discriminator. Conversely a low dropout rate leads to an initial stabilisation of G-D losses, but ultimately results in instability under the form of oscillation when training for a large number of batches. While training the different proposed DCGAN architectures, we did not observe mode collapse, indicating the DCGAN is less prone to a collapse compared to our *vanilla GAN*. @@ -117,7 +117,7 @@ We evaluate permutations of the architecture involving: \begin{figure} \begin{center} -\includegraphics[width=24em]{fig/med_cgan_ex.pdf} +\includegraphics[width=24em]{fig/med_cgan_ex.png} \includegraphics[width=24em]{fig/med_cgan.png} \caption{Medium CGAN} \label{fig:cmed} @@ -162,7 +162,7 @@ We observe increased accruacy as we increase the depth of the arhitecture at the \begin{figure} \begin{center} -\includegraphics[width=24em]{fig/smoothing_ex.pdf} +\includegraphics[width=24em]{fig/smoothing_ex.png} \includegraphics[width=24em]{fig/smoothing.png} \caption{One sided label smoothing} \label{fig:smooth} @@ -329,7 +329,7 @@ $$ L_{\textrm{total}} = \alpha L_{\textrm{LeNet}} + \beta L_{\textrm{generator}} \begin{figure} \begin{center} -\includegraphics[width=24em]{fig/short_dcgan_ex.pdf} +\includegraphics[width=24em]{fig/short_dcgan_ex.png} \includegraphics[width=24em]{fig/short_dcgan.png} \caption{Shallow DCGAN} \label{fig:dcshort} @@ -338,7 +338,7 @@ $$ L_{\textrm{total}} = \alpha L_{\textrm{LeNet}} + \beta L_{\textrm{generator}} \begin{figure} \begin{center} -\includegraphics[width=24em]{fig/long_dcgan_ex.pdf} +\includegraphics[width=24em]{fig/long_dcgan_ex.png} \includegraphics[width=24em]{fig/long_dcgan.png} \caption{Deep DCGAN} \label{fig:dclong} @@ -379,7 +379,7 @@ $$ L_{\textrm{total}} = \alpha L_{\textrm{LeNet}} + \beta L_{\textrm{generator}} \begin{figure} \begin{center} -\includegraphics[width=24em]{fig/short_cgan_ex.pdf} +\includegraphics[width=24em]{fig/short_cgan_ex.png} \includegraphics[width=24em]{fig/short_cgan.png} \caption{Shallow CGAN} \label{fig:cshort} @@ -388,7 +388,7 @@ $$ L_{\textrm{total}} = \alpha L_{\textrm{LeNet}} + \beta L_{\textrm{generator}} \begin{figure} \begin{center} -\includegraphics[width=24em]{fig/long_cgan_ex.pdf} +\includegraphics[width=24em]{fig/long_cgan_ex.png} \includegraphics[width=24em]{fig/long_cgan.png} \caption{Deep CGAN} \label{fig:clong} |