Linear least-squares Moore–Penrose inverse

the pseudoinverse provides least squares solution system of linear equations.



a
∈

m

(
m
,
n
;
k
)




{\displaystyle a\in \mathrm {m} (m,n;k)\,\!}

, given system of linear equations

a
x
=
b
,



{\displaystyle ax=b,\,}

in general, vector



x


{\displaystyle x}

solves system may not exist, or if 1 exist, may not unique. pseudoinverse solves least-squares problem follows:

∀
x
∈

k

n






{\displaystyle \forall x\in k^{n}\,\!}

, have



∥
a
x
−
b

∥

2


≥
∥
a
z
−
b

∥

2




{\displaystyle \|ax-b\|_{2}\geq \|az-b\|_{2}}





z
=

a

+


b


{\displaystyle z=a^{+}b}

,



∥
⋅

∥

2




{\displaystyle \|\cdot \|_{2}}

denotes euclidean norm. weak inequality holds equality if , if



x
=

a

+


b
+
(
i
−

a

+


a
)
w


{\displaystyle x=a^{+}b+(i-a^{+}a)w}

vector w; provides infinitude of minimizing solutions unless has full column rank, in case



(
i
−

a

+


a
)


{\displaystyle (i-a^{+}a)}

0 matrix. solution minimum euclidean norm



z
.


{\displaystyle z.}

this result extended systems multiple right-hand sides, when euclidean norm replaced frobenius norm. let



b
∈

m

(
m
,
p
;
k
)


{\displaystyle b\in \mathrm {m} (m,p;k)}

.

∀
x
∈

m

(
n
,
p
;
k
)




{\displaystyle \forall x\in \mathrm {m} (n,p;k)\,\!}

, have



∥
a
x
−
b

∥


f



≥
∥
a
z
−
b

∥


f





{\displaystyle \|ax-b\|_{\mathrm {f} }\geq \|az-b\|_{\mathrm {f} }}





z
=

a

+


b


{\displaystyle z=a^{+}b}

,



∥
⋅

∥


f





{\displaystyle \|\cdot \|_{\mathrm {f} }}

denotes frobenius norm.




^ penrose, roger (1956). on best approximate solution of linear matrix equations . proceedings of cambridge philosophical society. 52: 17–19. doi:10.1017/s0305004100030929. 
^ planitz, m., inconsistent systems of linear equations , mathematical gazette 63, october 1979, 181–185.