# Levenberg-Marquardt 演算法

Levenberg-Marquardt 演算法，又叫LM算法，係一種使用最細平方法求解非線性衲合問題數值最佳化演算法。個方法結合埋牛頓法特別係高斯-牛頓法佮正則化技術、強制遞減函數值嘅（譬如梯度下降法）。個演算法得名自Kenneth Levenberg同Donald Marquardt。

Levenberg-Marquardt 演算法頑健性明顯強過高斯-牛頓法，即使喺比較勩嘅起始條件下都高概率收斂得，但始終嘸保證會收斂得到。另外，喺近最細值嘅初始值時，通常會有啲慢。

## 描述

${\displaystyle \min _{x\in \mathbb {R} ^{m}}\|F(x)\|_{2}^{2}}$

${\displaystyle \min _{x\in \mathbb {R} ^{m}}\|F(x_{k})+J(x_{k})(x-x_{k})\|_{2}^{2}}$

J 係函數 F嘅Jacobi矩陣

${\displaystyle x_{k+1}=x_{k}-\alpha _{k}\left(J(x_{k})^{T}J(x_{k})+\Delta _{k}\right)^{-1}J(x_{k})^{T}F(x_{k})}$

${\displaystyle \min _{x\in \mathbb {R} ^{m}}\|F(x_{k})+J(x_{k})(x-x_{k})\|_{2}^{2}}$

${\displaystyle \min _{\|x-x_{k}\|

## 收斂

Levenberg-Marquardt 方法喺局部會變成高斯-牛頓法。因此個收斂係局部上線性嘅；甚至係二次嘅，喺接近最佳點嗰陣。

## 文獻

• K. Levenberg: A Method for the Solution of Certain Problems in Least Squares, Quart. Appl. Math. 2, 164-168, 1944.
• D. Marquardt: An Algorithm for Least-Squares Estimation of Nonlinear Parameters, SIAM J. Appl. Math. 11, 431-441, 1963.
• Jorge J. Moré: The Levenberg-Marquardt algorithm: Implementation and theory. In G. A. Watson (ed.): Numerical Analysis. Dundee 1977, Lecture Notes Math. 630, 1978, S. 105–116
• P. Gill, W. Murray und M. Wright: Practical Optimization, Academic Press 1981
• J. E. Dennis, Jr., und R. B. Schnabel: Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall Series in Computational Mathematics, Englewood Cliffs 1983

## 出面網頁

Levenberg-Marquardt 演算法啲實現自由使得嘅：