Add more details to CGAN

1 files changed, 23 insertions, 5 deletions

diff --git a/report/paper.md b/report/paper.md
index 364e6a5..03fad67 100644
--- a/report/paper.md
+++ b/report/paper.md
@@ -86,7 +86,9 @@ While training the different proposed DCGAN architectures, we did not observe mo
 
 ## CGAN Architecture description
 
-CGAN is a conditional version of a GAN which utilises labeled data. Unlike DCGAN, CGAN is trained with explicitly provided labels which allow CGAN to associate features with specific labels. This has the intrinsic advantage of allowing us to specify the label of generated data.  The baseline CGAN which we evaluate is visible in figure \ref{fig:cganrc}. The baseline GAN arhitecture presents a series blocks each contained a dense layer, ReLu layer and a Batch Normalisation layer. The baseline discriminator uses Dense layers, followed by ReLu and a Droupout layer.
+CGAN is a conditional version of a GAN which utilises labeled data. Unlike DCGAN, CGAN is trained with explicitly provided labels which allow CGAN to associate features with specific labels. This has the intrinsic advantage of allowing us to specify the label of generated data.  The baseline CGAN which we evaluate is visible in figure \ref{fig:cganrc}. The baseline CGAN arhitecture presents a series blocks each contained a dense layer, LeakyReLu layer and a Batch Normalisation layer. The baseline discriminator uses Dense layers, followed by LeakyReLu and a Droupout layer.
+
+The Convolutional CGAN analysed follows a structure similar to DCGAN and is presented in figure \ref{}. 
 
 We evaluate permutations of the architecture involving:
 
@@ -115,6 +117,22 @@ We evaluate permutations of the architecture involving:
 
 ## Tests on MNIST 
 
+When comparing the three levels of depth for the architectures it is possible to notice significant differences for the G-D losses balancing. In
+a shallow architecture we notice a high oscillation of the generator loss \ref{fig:}, which is being overpowered by the discriminator. Despite this we don't
+experience any issues with vanishing gradient, hence no mode collapse is reached. 
+Similarly, with a deep architecture the discriminator still overpowers the generator, and an equilibrium between the two losses is not acheived. The image quality in both cases is not really high: we can see that even after 20000 batches the some pictures appear to be slightly blurry \ref{fig:}.
+
+The three levels of dropout rates attempted do not affect the performance significantly, and as we can see in figures \ref{}, \ref{} and \ref{}, both
+image quality and G-D losses are comparable.
+
+The biggest improvement in performance is obtained through one-sided label smoothing, shifting the true labels form 1 to 0.9 to incentivize the discriminator.
+Using 0.1 instead of zero for the fake labels does not improve performance, as the discriminator loses incentive to do better. Performance results for
+one-sided labels smoothing with true labels = 0.9 are shown in figure \ref{}.
+
+ADD FORMULA?
+
+ADD VBN TALKING ABOUT TIME AND RESULTS
+
 \begin{figure}
 \begin{center}
 \includegraphics[width=24em]{fig/med_cgan_ex.png}
@@ -124,7 +142,7 @@ We evaluate permutations of the architecture involving:
 \end{center}
 \end{figure}
 
-### Inception Score
+# Inception Score
 
 Inception score is calculated as introduced by Tim Salimans et. al [@improved]. However as we are evaluating MNIST, we use LeNet-5 [@lenet]  as the basis of the inceptioen score.
 We use the logits extracted from LeNet:
@@ -146,8 +164,8 @@ Medium CGAN+LS        & 0.749    & 3.643         & 10:42        \\
 Convolutional CGAN+LS & 0.601    & 2.494         & 27:36        \\
 Medium CGAN DO=0.1    & 0.761    & 3.836         & 10:36        \\
 Medium CGAN DO=0.5    & 0.725    & 3.677         & 10:36        \\
-Medium CGAN+VBN       & 0.745    & 4.02          & 10:38        \\
-Medium CGAN+VBN+LS    & 0.783    & 4.31          & 10:38        \\
+Medium CGAN+VBN       & 0.745    & 4.02          & 19:38        \\
+Medium CGAN+VBN+LS    & 0.783    & 4.31          & 19:43        \\
 *MNIST original       & 0.9846   & 9.685         & N/A          \\ \hline
 \end{tabular}
 \end{table}
@@ -156,7 +174,7 @@ Medium CGAN+VBN+LS    & 0.783    & 4.31          & 10:38        \\
 
 ### Architecture
 
-We observe increased accruacy as we increase the depth of the arhitecture at the cost of the training time. There appears to be diminishing returns with the deeper networks, and larger improvements are achievable with specific optimisation technques.
+We observe increased accruacy as we increase the depth of the arhitecture at the cost of the training time. There appears to be diminishing returns with the deeper networks, and larger improvements are achievable with specific optimisation techniques.
 
 ### One Side Label Smoothing