diff options
author | Vasil Zlatanov <v@skozl.com> | 2018-12-14 11:45:28 +0000 |
---|---|---|
committer | Vasil Zlatanov <v@skozl.com> | 2018-12-14 11:45:28 +0000 |
commit | 1f174318840bfe4b2dcdafec0c7d74754afb6350 (patch) | |
tree | 0ecdd2d011f26b34dbf43dca45d0c9d0c77bfa4f /report | |
parent | 73248e8a915f0b9a85bf5cabde074164b0146690 (diff) | |
download | vz215_np1915-1f174318840bfe4b2dcdafec0c7d74754afb6350.tar.gz vz215_np1915-1f174318840bfe4b2dcdafec0c7d74754afb6350.tar.bz2 vz215_np1915-1f174318840bfe4b2dcdafec0c7d74754afb6350.zip |
Polish grammer of whole paper
Diffstat (limited to 'report')
-rw-r--r-- | report/paper.md | 60 |
1 files changed, 28 insertions, 32 deletions
diff --git a/report/paper.md b/report/paper.md index 7108b4b..a9a7388 100644 --- a/report/paper.md +++ b/report/paper.md @@ -116,24 +116,22 @@ improve identification accuracy, we consider it an additional baseline. ## Mahalanobis Distance -We were not able to achieve significant improvements using Mahalanobis for -original distance ranking compared to square euclidiaen metrics. +Mahalanobis distance is an extremely useful measure of distance between a point +(such as a query image feature vector) and a distribution (such as the gallery), which accounts for how many standard deviations away the point is from the mean of the distribution. It is unitless and scale invariant, in a way standartising the data. -The Mahalanobis distance metric was used to create the ranklist as an alternative to euclidean distance: +We used Mahalanobis distance as an altenrative metric to Euclidean distance: $$ d_M(p,g_i) = (p-g_i)^TM(p-g_i). $$ -When performing Mahalanobis with the covariance matrix $M$ generated from the training set, reported accuracy is reduced to 38%. +We were not able to achieve significant improvements using Mahalanobis for original distance ranking compared to square euclidiaen metrics. When performed with the covariance matrix $M$ generated from the training set, reported accuracy is reduced to 38%. -We also attempted to perform the same Mahalanobis metric on a reduced PCA featureset. PCA performed with the top 100 eigenvectors -reduces the feature space while keeping 94% of the data variance. This allowed for significant execution +We also attempted to perform the same Mahalanobis metric on a reduced PCA featureset. PCA performed with the top 100 eigenvectors reduces the feature space while keeping 94% of the data variance. This allowed for significant execution time improvements due to the greatly reduced computation requierments for smaller featurespace, but nevertheless demonstrated no improvements over an euclidean metric. -These results are likely due to the extremely low covariance of features in the training set. -This is evident when looking at the covariance matrix of the training data, and is also -visible in figure \ref{fig:subspace}. This is likely the result of the feature -transformations performed by the ResNet-50 convolution model from which features were extracted. +These results are likely due to the low covariance between features in the training set. This is evident when looking at the covariance matrix of the training data, and is also visible in figure \ref{fig:subspace}. This is likely the result of the feature transformations performed by the ResNet-50 convolution model from which features were extracted. + +A connection can also be made to the standardisation baseline previously evaluated and the reduction of re-identification accuracy associated with it. \begin{figure} \begin{center} @@ -144,16 +142,15 @@ transformations performed by the ResNet-50 convolution model from which features \end{center} \end{figure} -While we did not use Mahalanobis as a primary distance metric, it is possible to use the Mahalanobis metric, -together with the next investigated solution involving $k$-reciprocal re-ranking. +While we did not use Mahalanobis as a primary distance metric, it is possible to beneficially use the Mahalanobis metric on differently processed feature sets, together with the next investigated solution involving $k$-reciprocal re-ranking. # Suggested Improvement ## $k$-reciprocal Re-ranking Formulation -The approach addressed to improve the identification performance is based on -$k$-reciprocal re-ranking. The following section summarizes the idea behind -the method illustrated in reference @rerank-paper. +The approach investigated for improving identification performance is based on +$k$-reciprocal re-ranking. The following section summarizes the technique +introduced in reference @rerank-paper. We define $N(p,k)$ as the top $k$ elements of the rank-list generated through NN, where $p$ is a query image. The k reciprocal rank-list, $R(p,k)$ is defined as the @@ -183,8 +180,7 @@ e\textsuperscript{\textit{-d(p,g\textsubscript{i})}}, & \text{if}\ \textit{g\tex \end{cases} \end{equation} -Through this transformation it is possible to reformulate the distance obtained -through Jaccardian metric as: +Through this transformation it is possible to reformulate the distance with the Jaccardian metric as: $$ d_J(p,g_i)=1-\frac{\sum\limits_{j=1}^N min(V_{p,g_j},V_{g_i,g_j})}{\sum\limits_{j=1}^N max(V_{p,g_j},V_{g_i,g_j})}. $$ @@ -195,18 +191,21 @@ We refer to $k_2$ since it allows constricting the the size of the neighbors to from the $k_2$ neighbors. The dimension k of the *$R^*$* set will instead be defined as $k_1$: $R^*(g_i,k_1)$. -The distances obtained are then mixed, obtaining a final distance $d^*(p,g_i)$ that is used to obtain the +The distances obtained are then combined, obtaining a final distance $d^*(p,g_i)$ that is used to obtain the improved rank-list: $d^*(p,g_i)=(1-\lambda)d_J(p,g_i)+\lambda d(p,g_i)$. ## Optimisation -The aim is to learn optimal values for $k_1,k_2$ and $\lambda$ in the training set that improve top1 identification accuracy. -This is done through a simple multi-direction search algorithm followed by exhaustive search to estimate -$k_{1_{opt}}$ and $k_{2_{opt}}$ for eleven values of $\lambda$ from zero (only Jaccard distance) to one (only original distance) + +The goal is to learn optimal values for $k_1,k_2$ and $\lambda$ using the training set, +such that an improved Top-1 identification accuracy can be found. +This is done using a multi-direction search algorithm to estimate $k_{1_{opt}}$ and $k_{2_{opt}}$ +and an exhaustive search for $\lambda$ from $ \lambda = 0 $ (exclusively Jaccard distance) to $ \lambda = 1 $ (only original distance) in steps of 0.1. The results obtained through this approach suggest: $k_{1_{opt}}=9, k_{2_{opt}}=3, 0.1\leq\lambda_{opt}\leq 0.3$. -It is possible to verify that the optimisation of $k_{1_{opt}}$, $k_{2_{opt}}$ and $\lambda$ -has been successful. Figures \ref{fig:pqvals} and \ref{fig:lambda} show that the optimal values obtained from -training are close to the ones for the local maximum of gallery and query. +To verify optimisation of $k_{1_{opt}}$, $k_{2_{opt}}$ heat plots were performed heat on +figures \ref{fig:pqvals}, showing that the optimal values obtained from +training are close to the ones for the local maximum of gallery and query. +The $\lambda$ search is plotted on figure \ref{fig:lambda}. \begin{figure} \begin{center} @@ -228,13 +227,12 @@ training are close to the ones for the local maximum of gallery and query. ## $k$-reciprocal Re-ranking Evaluation -Re-ranking achieves better results than the other baseline methods analyzed both as top $k$ -accuracy and mean average precision. +Re-ranking achieves better results than the other baseline methods comparing both top $k$ accuracy and mean average precision (mAP). It is also necessary to estimate how precise the rank-list generated is. For this reason an additional method of evaluation is introduced: mAP. See reference @mAP. -It is possible to see in figure \ref{fig:ranklist2} how the rank-list generated for the same five queries of figure \ref{fig:eucrank} -has improved for the fifth query. The mAP improves from 47.2% to 61.7%. +The rank-list improvement over Euclidean NN on \ref{fig:eucrank} with re-ranking is shown visually in \ref{fig:ranklist2}. With re-ranking +mAP improves from 47.2% to 61.7%. \begin{figure} \begin{center} @@ -246,7 +244,7 @@ has improved for the fifth query. The mAP improves from 47.2% to 61.7%. Figure \ref{fig:compare} shows a comparison between top $k$ identification accuracies obtained with the two methods. It is noticeable that the $k$-reciprocal re-ranking method significantly -improves the results even for top$1$, boosting identification accuracy from 47% to 56.5%. +improves the results even for top-$1$, boosting identification accuracy from 47% to 56.5%. The difference between the top $k$ accuracies of the two methods gets smaller as we increase $k$. \begin{figure} @@ -257,7 +255,7 @@ The difference between the top $k$ accuracies of the two methods gets smaller as \end{center} \end{figure} -Figure \ref{fig:rerank} shows successful reranking as performed using the query and gallery sets. The $k$-reciprocal ranklist of the second nearest neighbour, contains the target identity, and when the final distance is calculated as a function original and Jaccard distance, it becomes the closest neighbour. This happens with three cases of the identity, despite the image visually hard to classify due to the presence of another person in the photos and a completely different pose. +Figure \ref{fig:rerank} shows successful re-ranking as performed using the query and gallery sets. The $k$-reciprocal ranklist of the second original nearest neighbour, contains the target identity, and when the final distance is calculated as a function original and Jaccard distance, the matching identity becomes the closest neighbour. This happens with three cases of the identity, despite the image visually hard to classify due to the presence of another person in the photos and a completely different pose. \begin{figure} \begin{center} @@ -283,8 +281,6 @@ Re-ranking with Jaccard & \textit{O(N\textsuperscript{2}logN)} \\ \hl \label{tab:complexity} \end{table} - - Overall the re-ranking method gives a significant improvement to both top $k$ accuracy and mean average precision. The cost of this operation is an increase in computation time due to the change in complexity from the baseline, summarized in the table. |