# Forulation of the Addresssed Machine Learning Problem ## Probelm Definition The person re-identification problem presented in this paper requires matching pedestrian images from disjoint cameras by pedestrian detectors. This problem is challenging, as identities captured in photsos are subject to various lighting, pose, blur, background and oclusion from various camera views. This report considers features extracted from the CUHK03 dataset, following a 50 layer Residual network (Resnet50). This paper considers distance metrics techniques which can be used to perform person re-identification across *disjoint* cameras, using these features. ## Dataset - CUHK03 Summary The dataset CUHK03 contains 14096 pictures of people captured from two different cameras. The feature vectors used, extracted from a trained ResNet50 model , contain 2048 features that are used for identification. The pictures represent 1467 different identities, each of which appears 9 to 10 times. Data is seperated in train, query and gallery sets with `train_idx`, `query_idx` and `gallery_idx` respectively, where the training set has been used to develop the ResNet50 model used for feature extraction. This procedure has allowed the evaluation of distance metric learning techniques on the query and gallery sets, with the knowledge that we are not comparing overfitted features, as they were extracted based on the model derived from the training set. ## Probelm to solve The problem to solve is to create a ranklist for each image of the query set by finding the nearest neighbor(s) within a gallery set. However gallery images with the same label and taken from the same camera as the query image should not be considered when forming the ranklist. ## Nearest Neighbor ranklist Nearest Neighbor aims to find the gallery image whose feature are the closest to the ones of a query image, predicting the class of the query image as the same of its nearest neighbor(s). The distance between images can be calculated through different distance metrics, however one of the most commonly used is euclidean distance: $$ \textrm{NN}(x) = \operatorname*{argmin}_{i\in[m]} \|x-x_i\|^2 $$ *Square root when calculating euclidean distance is ommited as it does not affect ranking by distance* Alternative distance metrics exist such as jaccardian and mahalanobis, which can be used as an alternative to euclidiean distance. # Baseline Evaluation To evaluate improvements brought by alternative distance learning metrics a baseline is established through nearest neighbour identification as previously described. Identification accuracies at top1, top5 and top10 are respectively 47%, 67% and 75% (figure \ref{fig:baselineacc}). The mAP for a ranklist of size 10 is 33.3%. \begin{figure} \begin{center} \includegraphics[width=20em]{fig/baseline.pdf} \caption{Recognition accuracy of baseline Nearest Neighbor @rank k} \label{fig:baselineacc} \end{center} \end{figure} Figure \ref{fig:eucrank} shows the ranklist generated through baseline NN for 5 query images(black). Correct identification is shown in green and incorrect identification is shown in red. \begin{figure} \begin{center} \includegraphics[width=22em]{fig/eucranklist.png} \caption{Ranklist @rank10 generated for 5 query images} \label{fig:eucrank} \end{center} \end{figure} Magnitude normalization of the feature vectors does not improve accuracy results of the baseline as it can be seen in figure \ref{fig:baselineacc}. This is due to the fact that the feature vectors appear scaled, releative to their significance, for optimal distance classification, and as such normalising loses this scaling by importance which has previously been introduced to the features. ## kMean Clustering An addition considered for the baseline is *kMeans clustering*. In theory this method allows to reduce computational complexity of the baseline NN by forming clusters and performing a comparison between query image and clusters centers. The elements associated with the closest cluster center are then considered to perform NN and classify the query image. This method did not bring any major improvement to the baseline, as it can be seen from figure \ref{fig:baselineacc}. It is noticeable how the number of clusters affects performance, showing better identification accuracy for a number of clusters away from the local minimum achieved at 60 clusters (figure \ref{fig:kmeans}). This trend can likely be explained by the number of distance comparison's performed. We would expect clustering with $k=1$ and $k=\textrm{label count}$ to have the same performance the baseline approach without clustering, as we are performing the same number of comparisons. Clustering is a great method of reducing computation time. Assuming 39 clusters of 39 neighbours we would be performing only 78 distance computation for a gallery size of 1487, instead of the original 1487. This however comes at the cost of ignoring neighbours from other clusters which may be closer. Since clusters do not necessarily have the same number of datapoints inside them (sizes are uneven), we find that the lowest average number of comparison happens at around 60 clusters, which also appears to be the worst performing number of clusters. We find that for the query and gallery set clustering does not seem to improve identification accuracy, and consider it an additional baseline. \begin{figure} \begin{center} \includegraphics[width=17em]{fig/kmeanacc.pdf} \caption{Top 1 Identification accuracy varying kmeans cluster size} \label{fig:kmeans} \end{center} \end{figure} # Suggested Improvement ## k-reciprocal Reranking \begin{figure} \begin{center} \includegraphics[width=24em]{fig/ranklist.png} \caption{Ranklist (improved method) @rank10 generated for 5 query images} \label{fig:ranklist2} \end{center} \end{figure} \begin{figure} \begin{center} \includegraphics[width=20em]{fig/comparison.pdf} \caption{Comparison of recognition accuracy @rank k (KL=0.3,K1=9,K2=3)} \label{fig:compare} \end{center} \end{figure} \begin{figure} \begin{center} \includegraphics[width=12em]{fig/pqvals.pdf} \includegraphics[width=12em]{fig/trainpqvals.pdf} \caption{Identification accuracy varying K1 and K2 (gallery-query left, train right)} \label{fig:pqvals} \end{center} \end{figure} \begin{figure} \begin{center} \includegraphics[width=12em]{fig/lambda_acc.pdf} \includegraphics[width=12em]{fig/lambda_acc_tr.pdf} \caption{Top 1 Identification Accuracy with Rerank varying lambda(gallery-query left, train right)} \label{fig:lambda} \end{center} \end{figure} # Comment on Mahalnobis Distance as a metric We were not able to achieve significant improvements using mahalanobis for original distance ranking compared to square euclidiaen metrics. Results can be observed using the `-m|--mahalanobis` when running evalution with the repository complimenting this paper. It has been verified that the inverse covariance matrix used is PSD, satisfying $d_a(x_1,x_2) \geq 0$. # Conclusion # References # Appendix \begin{figure} \begin{center} \includegraphics[width=17em]{fig/cdist.pdf} \caption{First two features of gallery(o) and query(x) feature data} \label{fig:subspace} \end{center} \end{figure} \begin{figure} \begin{center} \includegraphics[width=17em]{fig/clusteracc.pdf} \caption{Top k identification accuracy for cluster count} \label{fig:clustk} \end{center} \end{figure} \begin{figure} \begin{center} \includegraphics[width=17em]{fig/jaccard.pdf} \caption{Explained Jaccard} \label{fig:jaccard} \end{center} \end{figure} \begin{figure} \begin{center} \includegraphics[width=17em]{fig/mahalanobis.pdf} \caption{Explained Mahalanobis} \label{fig:mahalanobis} \end{center} \end{figure}