MODIFIED ARMIJO RULE ON GRADIENT DESCENT AND CONJUGATE GRADIENT

Abstract

Armijo rule is an inexact line search method to determine step size in some descent method to solve unconstrained local optimization. Modified Armijo was introduced to increase the numerical performance of several descent algorithms that applying this method. The basic difference of Armijo and its modified are in existence of a parameter and estimating the parameter that is updated in every iteration. This article is comparing numerical solution and time of computation of gradient descent and conjugate gradient hybrid Gilbert-Nocedal (CGHGN) that applying modified Armijo rule. From program implementation in Matlab 6, it's known that gradient descent was applying modified Armijo more effectively than CGHGN from one side: iteration needed to reach some norm of the gradient  (input by the user). The amount of iteration was representing how long the step size of each algorithm in each iteration. In another side, time of computation has the same conclusion.