スキップしてメイン コンテンツに移動

Mahalanobis' Distance

Introduction


Today, I will write about Mahalanobis’ Distance.
Mahalanobis’ Distance is used when each dimension has a relationship.
This distance is fulfilled definition of distance.
Mahalanobis’ Distance is important for Statics.
If you interested in Statics or Machine Learning, Please see my this blog.

Overview

  • definition of distance
  • deficition of Mahalanobis’ Distance
  • image of Mahalanobis’ Distance

definition of distance

if d is distance function, d if fulfilled following condtion.
\(d:X \times X -> R\)
  • \(d(x,y) \geq 0\)
  • \(d(x,y) = 0 \leftrightarrow x = y\)
  • \(d(x,y) = d(y,x)\)
  • \(d(x,z) \leq d(x,y) + d(y,z)\)

Mahalanobis’ Distance

Mahalanobis’ Distance is distance function.
Mahalanobis’ Distance is defined by following from
\[D_{M}(x) = \sqrt{(x-\mu)^T \Sigma^{-1} (x-\mu)}\]
here, \(\mu\) is mean vector
\[\mu = (\mu_1,\mu_2,....,\mu_n)\]
and, \(\Sigma\) is variance-convariance matrix.
Mahalanobis’ Distance between x and y is
\begin{eqnarray*} d(x,y) &=& \sqrt{(x-\mu-(y-\mu)^T \Sigma^{-1} (x-\mu-(y-\mu)}\\ &=& \sqrt{(x-y)^T \Sigma^{-1} (x-y)} \end{eqnarray*}

Image of Mahalanobis’ Distance

first, I think of Eculid distance. Eculid distance is following form.
\[d(x,y) = \sqrt{<x^T,y>}\]
The eculid distance regard distance between \(x\) and \(y\) as same if x and y exists over same circle.
enter image description here
This thinking has reason that data distributed like following.
enter image description here
but ,if data distributed like ellipse, it is not good at to use Euclid distance.
enter image description here
Because I want to regard distance between X and Y as same.
Mahalanobis’ Distance is regard distance between X and Y as same if X and Y have existed over the same ellipse.
enter image description here
Distance is always used Machine Learning. Machine Learning use Eculid distance, but We get interesting result by using Mahalanobis’ Distance.
Labels:

コメント

コメントを投稿

このブログの人気の投稿

ヘッセ行列

Introduction English ver 今日は、ヘッセ行列を用いたテイラー展開について書こうと思います。 これは最適化を勉強するにあたって、とても大事になってくるので自分でまとめて残しておくことにしました。とくに、機械学習では最適化を必ず行うため、このブログのタイトルにもマッチした内容だと思います。 . 概要 ヘッセ行列の定義 ベクトルを用いたテイラー展開 関数の最適性 ヘッセ行列の定義 仮定 f は次のような条件を満たす関数です。. f はn次元ベクトルから実数値を出力します。 このベクトルは次のように表せます。 \[x = [x_1,x_2,,,,x_n]\] \(\forall x_i , i \in {1,2,,,n}\), f は二回偏微分可能です。 定義 ヘッセ行列は \(\frac{\partial^2}{\partial x_i \partial x_j}を (i,j)要素に持ちます。\) よってヘッセ行列は次のように表せます。 \[ H(f) = \left( \begin{array}{cccc} \frac{\partial^ 2}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} & &\ldots \frac{\partial^2 f}{\partial x_1 \partial x_n} \\ \frac{\partial^ 2 f}{\partial x_1 \partial x_2} & \frac{\partial^ 2 f}{\partial x_2^ 2} & \ldots & \frac{\partial^2 f}{\partial x_2 \partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^ 2 f}{\partial x_n \partial x_2} & \frac{\partial^ 2 f}{\partial x_n \partial x_2} & \ldo...
コメントを投稿
続きを読む

Plane in two dimention

Introduction 日本語 ver Today, I prove this theorem. Plane in two dimention is expressed following. \[\{x|<x,v> = 0\}\] however, v is orthogonal vector for plane and not zero vector. Proof \[\forall k \in \{x|<x,v> = 0\},\] k is fulfill this form. \[<k,v> = 0\] Now, because k and v in two dimentinal space, each vector express following. \[k = (k_1,k_2)\] \[v = (v_1,v_2)\] Thus, \(<k,v>=k_1v_1 + k_2v_2=0\) Change this equation. \[k_2 = -\frac{v_1}{v_2} k_1\] This equation is plane that slope is \(-\frac{v_1}{v_2}\). Q.E.D
コメントを投稿
続きを読む

Rolle’s theorem

Introduction 日本語 ver This post is written Rolle’s theorem. The mean-value theorem is proved by Rolle’s theorem. I will write Mean-value theorem at a later. I introduce Maximum principle because proving Rolle’s theorem need Maximum principle. Maximum principle It is very easy. f is continuous function on bounded closed interval.\(\implies\)** f have max value.** Proof This proof is difficult. I write this proof in other posts. Maximum Principle Rolle’s theorem f is continuous function on [a,b] and differentiable function on (a,b). \[f(a) = f(b) \implies \exists ~~c ~~s.t~~ f'(c) = 0 , a<c<b\] Proof f(x) is constant function \[\forall c \in (a,b) , f'(c) = 0\] else when \(\exists t ~~s.t~~f(a) < f(t)\), \(\exists c ~~s.t~~ \max f(x) = f(c)\) by Maximum principle I proof \(f'(c)=0\) f is differentiable on \(x = c\) and \(f(c) >= f(c+h)\). Thus \[f'(c) = \lim_{h \rightarrow +0} \frac{f(c+h) - f(c)}{h} \leq 0\] \[f'(c) = \lim...
コメントを投稿
続きを読む