You have a model with parameters
Using the multivariate Taylor series, we can update the initial guess
This gives
This iterative procedure is nice because the functions involved often have simple forms (e.g in logistic regression,
Monday, April 28, 2008
Iterative parameter finding
I should be getting ready for my viva, but instead, I reread this cool bit on how to find the best parameters for a model.
You have a model with parameters
, and you want to fit the parameters to the data 
. For the maximum log-likelihood, you would find the zero of the derivative  = \nabla ln p(D | \underline \theta))
. But suppose that is nontrivial. Then you can use the Newton-Rhapson method for iterative parameter finding.
Using the multivariate Taylor series, we can update the initial guess
for the best parameters.
 = f(\hat{ \underline \theta}) + \nabla f (\underline{ \hat \theta})(\underline \theta - \hat {\underline \theta}) + )
small terms
This gives)^{-1} f(\underline {\hat \theta}))
This iterative procedure is nice because the functions involved often have simple forms (e.g in logistic regression, \underline x_n)
is a combination of inputs that give classification errors), and it gives a solution in cases that are not analytically tractable.
You have a model with parameters
Using the multivariate Taylor series, we can update the initial guess
This gives
This iterative procedure is nice because the functions involved often have simple forms (e.g in logistic regression,
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment