Mahalanobis' Distance

Introduction

日本語 ver

Today, I will write about Mahalanobis’ Distance.
Mahalanobis’ Distance is used when each dimension has a relationship.
This distance is fulfilled definition of distance.
Mahalanobis’ Distance is important for Statics.
If you interested in Statics or Machine Learning, Please see my this blog.

Overview

• definition of distance
• deficition of Mahalanobis’ Distance
• image of Mahalanobis’ Distance

definition of distance

if d is distance function, d if fulfilled following condtion.
$d:X \times X -> R$

• $d(x,y) \geq 0$
• $d(x,y) = 0 \leftrightarrow x = y$
• $d(x,y) = d(y,x)$
• $d(x,z) \leq d(x,y) + d(y,z)$

Mahalanobis’ Distance

Mahalanobis’ Distance is distance function.
Mahalanobis’ Distance is defined by following from
$$D_{M}(x) = \sqrt{(x-\mu)^T \Sigma^{-1} (x-\mu)}$$
here, $\mu$ is mean vector
$$\mu = (\mu_1,\mu_2,....,\mu_n)$$
and, $\Sigma$ is variance-convariance matrix.
Mahalanobis’ Distance between x and y is
$$\begin{eqnarray*} d(x,y) &=& \sqrt{(x-\mu-(y-\mu)^T \Sigma^{-1} (x-\mu-(y-\mu)}\\ &=& \sqrt{(x-y)^T \Sigma^{-1} (x-y)} \end{eqnarray*}$$

Image of Mahalanobis’ Distance

first, I think of Eculid distance. Eculid distance is following form.
$$d(x,y) = \sqrt{<x^T,y>}$$
The eculid distance regard distance between $x$ and $y$ as same if x and y exists over same circle.

This thinking has reason that data distributed like following.

but ,if data distributed like ellipse, it is not good at to use Euclid distance.

Because I want to regard distance between X and Y as same.
Mahalanobis’ Distance is regard distance between X and Y as same if X and Y have existed over the same ellipse.

Distance is always used Machine Learning. Machine Learning use Eculid distance, but We get interesting result by using Mahalanobis’ Distance.