How to perform nonlinear regression?

By Indra Giri & Priya Chetty on October 30, 2017

Regression analysis is a statistical tool to study the relationship between variables. These variables are the outcome variable and one or more exposure variables. In other words, regression analysis is an equation which predicts a response from the value of a certain predictor. In a linear regression analysis model the regression function µY(X1…….Xk) is a linear function of the unknown parameters. However, in a nonlinear regression model, the regression function is not a linear function of the unknown parameters. Therefore nonlinear regression is a regression in which the dependent variables are modeled as a non-linear function of model parameters and one or more independent variables.

Nonlinear regression equation

Ordinary Least Square (OLS) is a commonly used assumption to estimating the unknown parameters in a nonlinear regression function. The most commonly used set of assumptions for nonlinear is the same as assumptions for linear regression. However the only exception is that the regression function µY(X1…….Xk) is a nonlinear function of the unknown parameters instead of a linear function of the parameter. Assumptions for nonlinear regression are satisfied with µy(x) given by the following equation:

µy(x) = β12e3x  (1)

Where β1, β2, and β3 are unknown parameters and researcher wants to estimate β1, β2, and β3. So, assume that ‘β1 = 0.0, β2= 2.0 and β3 = 0.5. µy’ is the function of X in the equation 1. Also assume that this equation shows a relation between µy and X exponentially. Hence the equation shows a nonlinear relation. Since µy is not a linear combination of the independent variables, it is a non-linear regression.

Example of nonlinear regression

Theory

This section explains the application of nonlinear regression test through a case study. Suppose that in the sample dataset there is an independent variable (x), and variable y. Variable y is the squared value of variable x.

The equation for the simplest nonlinear function is y = x2. The table below shows the function of x and y values.

Table 1: Sample dataset for nonlinear regression test

On a graph, these values form a curved, U-shaped line called a parabola.

Image 1: Graph of nonlinear regression
Image 1: Graph of nonlinear regression

The equation for a quadratic function is y = x2. But other quadratic function formulas are more complex. Some of the quadratic equations are as follows:

y = ax2 + bx + c

y = (ax + b)(cx + d)

y = a(x+b)2+ c

Here in each equation, the independent variable is multiplied by itself. The letters a, b, c, and d are coefficients—their presence in the equation modifies the shape and location of the parabola.

Practical example

Here, a sample dataset is used to explain nonlinear regression using SPSS software. The dataset consists of anthropometry data with the variables BMI, Height and Weight. While ‘BMI’ is dependent variable, ‘Height’ and ‘Weight’ are independent variables. The aim is to examine the impact of height and weight on BMI. The impact is measured by the regression coefficients for each independent variables. The first step is to plot the data to arrive at these initial guesses.

Table 4.1 shows the iteration history. Here, iteration history shows the progress of the clustering process at each step and also estimates the summary statistics in nonlinear regression. Therefore iteration history goes to 4 steps starting with the initial value B1 is 0, B2 is 1 and B3 is 1. This is to get a best fit result with the value of B1 equal -24.6, B2 equal 27.8 and B3 equal -.027.

MODEL PROGRAM B1=0 B2=1 B3=1.

COMPUTE PRED_= B1 + B2 * HEIGHTM + B3 * WEIGHTKG.

NLR BMI

Table 2: Nonlinear regression results of sample dataset
Table 1: Nonlinear regression results of sample dataset

Software that support nonlinear regression test are R, SAS, MATLAB, STATA and SPSS.

Discuss

1 thought on “How to perform nonlinear regression?”