Mahalanobis' Distance

Introduction

日本語 ver

Today, I will write about Mahalanobis’ Distance.
Mahalanobis’ Distance is used when each dimension has a relationship.
This distance is fulfilled definition of distance.
Mahalanobis’ Distance is important for Statics.
If you interested in Statics or Machine Learning, Please see my this blog.

Overview

• definition of distance
• deficition of Mahalanobis’ Distance
• image of Mahalanobis’ Distance

definition of distance

if d is distance function, d if fulfilled following condtion.
\(d:X \times X -> R\)

• \(d(x,y) \geq 0\)
• \(d(x,y) = 0 \leftrightarrow x = y\)
• \(d(x,y) = d(y,x)\)
• \(d(x,z) \leq d(x,y) + d(y,z)\)

Mahalanobis’ Distance

Mahalanobis’ Distance is distance function.
Mahalanobis’ Distance is defined by following from
\[D_{M}(x) = \sqrt{(x-\mu)^T \Sigma^{-1} (x-\mu)}\]
here, \(\mu\) is mean vector
\[\mu = (\mu_1,\mu_2,....,\mu_n)\]
and, \(\Sigma\) is variance-convariance matrix.
Mahalanobis’ Distance between x and y is
\begin{eqnarray*} d(x,y) &=& \sqrt{(x-\mu-(y-\mu)^T \Sigma^{-1} (x-\mu-(y-\mu)}\\ &=& \sqrt{(x-y)^T \Sigma^{-1} (x-y)} \end{eqnarray*}

Image of Mahalanobis’ Distance

first, I think of Eculid distance. Eculid distance is following form.
\[d(x,y) = \sqrt{<x^T,y>}\]
The eculid distance regard distance between \(x\) and \(y\) as same if x and y exists over same circle.

This thinking has reason that data distributed like following.

but ,if data distributed like ellipse, it is not good at to use Euclid distance.

Because I want to regard distance between X and Y as same.
Mahalanobis’ Distance is regard distance between X and Y as same if X and Y have existed over the same ellipse.

Distance is always used Machine Learning. Machine Learning use Eculid distance, but We get interesting result by using Mahalanobis’ Distance.