How to perform nonlinear regression?

By Indra Giri & Priya Chetty on October 30, 2017

Regression analysis is a statistical tool for studying the relationship between variables, which are the outcome variable and one or more exposure variables. In other words, regression analysis is an equation that predicts a response from the value of a certain predictor. In a linear regression analysis model, the regression function µY(X1…….Xk) is a linear function of the unknown parameters.

However, in a nonlinear regression model, the regression function is not a linear function of the unknown parameters. Therefore, nonlinear regression is a regression in which the dependent variables are modeled as a non-linear function of model parameters and one or more independent variables.

Nonlinear regression equation

Ordinary Least Square (OLS) is a commonly used assumption to estimate the unknown parameters in a nonlinear regression function. The most commonly used set of assumptions for nonlinear is the same as the assumptions for linear regression. However, the only exception is that the regression function µY(X1…….Xk) is a nonlinear function of the unknown parameters instead of a linear function. Assumptions for nonlinear regression are satisfied with µy(x) given by the following equation:

µy(x) = β1+β2e-β3x  (1)

Where β1, β2, and β3 are unknown parameters and researcher wants to estimate β1, β2, and β3. So, assume that ‘β1 = 0.0, β2= 2.0 and β3 = 0.5. µy’ is the function of X in the equation 1. Also assume that this equation shows a relation between µy and X exponentially. Hence the equation shows a nonlinear relation. Since µy is not a linear combination of the independent variables, it is a non-linear regression.

Example of nonlinear regression

Theory

This section explains the application of the nonlinear regression test through a case study. Suppose that in the sample dataset there is an independent variable (x), and variable y. Variable y is the squared value of variable x.

The equation for the simplest nonlinear function is y = x2. The table below shows the function of x and y values.

Table 1: Sample dataset for nonlinear regression test

On a graph, these values form a curved, U-shaped line called a parabola.

Image 1: Graph of nonlinear regression
Image 1: Graph of nonlinear regression

The equation for a quadratic function is y = x2. But other quadratic function formulas are more complex. Some of the quadratic equations are as follows:

y = ax2 + bx + c

y = (ax + b)(cx + d)

y = a(x+b)2+ c

In each equation, the independent variable is multiplied by itself. The letters a, b, c, and d are coefficients—their presence in the equation modifies the shape and location of the parabola.

Practical example

Here, a sample dataset is used to explain nonlinear regression using SPSS software. The dataset consists of anthropometry data with the variables BMI, Height and Weight. While ‘BMI’ is the dependent variable, ‘Height’ and ‘Weight’ are independent variables. The aim is to examine the impact of height and weight on BMI. The impact is measured by the regression coefficients for each independent variable. The first step is to plot the data to arrive at these initial guesses.

Table 4.1 shows the iteration history. Here, iteration history shows the progress of the clustering process at each step and also estimates the summary statistics in nonlinear regression. Therefore, the iteration history goes to 4 steps, starting with the initial value B1 is 0, B2 is 1, and B3 is 1. This is to get a best fit result with the value of B1 equal to -24.6, B2 equal to 27.8 and B3 equal to -.027.

MODEL PROGRAM B1=0 B2=1 B3=1
COMPUTE PRED_= B1 + B2 * HEIGHTM + B3 * WEIGHTKG
NLR BMI
Table 2: Nonlinear regression results of sample dataset
Table 1: Nonlinear regression results of sample dataset

Software that supports nonlinear regression tests are R, SAS, MATLAB, STATA and SPSS.

I am a management graduate with specialisation in Marketing and Finance. I have over 12 years' experience in research and analysis. This includes fundamental and applied research in the domains of management and social sciences. I am well versed with academic research principles. Over the years i have developed a mastery in different types of data analysis on different applications like SPSS, Amos, and NVIVO. My expertise lies in inferring the findings and creating actionable strategies based on them. 

Over the past decade I have also built a profile as a researcher on Project Guru's Knowledge Tank division. I have penned over 200 articles that have earned me 400+ citations so far. My Google Scholar profile can be accessed here. 

I now consult university faculty through Faculty Development Programs (FDPs) on the latest developments in the field of research. I also guide individual researchers on how they can commercialise their inventions or research findings. Other developments im actively involved in at Project Guru include strengthening the "Publish" division as a bridge between industry and academia by bringing together experienced research persons, learners, and practitioners to collaboratively work on a common goal. 

 

Discuss

1 thought on “How to perform nonlinear regression?”