How to perform nonlinear regression?

Regression analysis is a statistical tool to study the relationship between variables. These variables are the outcome variable and one or more exposure variables. In other words, regression analysis is an equation which predicts a response from the value of a certain predictor. In a linear regression analysis model the regression function µY(X1…….Xk) is a linear function of the unknown parameters. However, in a nonlinear regression model, the regression function is not a linear function of the unknown parameters. Therefore nonlinear regression is a regression in which the dependent variables are modeled as a non-linear function of model parameters and one or more independent variables.

Nonlinear regression equation

Ordinary Least Square (OLS) is a commonly used assumption to estimating the unknown parameters in a nonlinear regression function. The most commonly used set of assumptions for nonlinear is the same as assumptions for linear regression. However the only exception is that the regression function µY(X1…….Xk) is a nonlinear function of the unknown parameters instead of a linear function of the parameter. Assumptions for nonlinear regression are satisfied with µy(x) given by the following equation:

µy(x) = β12e3x  (1)

Where β1, β2, and β3 are unknown parameters and researcher wants to estimate β1, β2, and β3. So, assume that ‘β1 = 0.0, β2= 2.0 and β3 = 0.5. µy’ is the function of X in the equation 1. Also assume that this equation shows a relation between µy and X exponentially. Hence the equation shows a nonlinear relation. Since µy is not a linear combination of the independent variables, it is a non-linear regression.

Example of nonlinear regression

Theory

This section explains the application of nonlinear regression test through a case study. Suppose that in the sample dataset there is an independent variable (x), and variable y. Variable y is the squared value of variable x.

The equation for the simplest nonlinear function is y = x2. The table below shows the function of x and y values.

Table 1: Sample dataset for nonlinear regression test

 

On a graph, these values form a curved, U-shaped line called a parabola.

Image 1: Graph of nonlinear regression

Image 1: Graph of nonlinear regression

The equation for a quadratic function is y = x2. But other quadratic function formulas are more complex. Some of the quadratic equations are as follows:

y = ax2 + bx + c

y = (ax + b)(cx + d)

y = a(x+b)2+ c

Here in each equation, the independent variable is multiplied by itself. The letters a, b, c, and d are coefficients—their presence in the equation modifies the shape and location of the parabola.

Practical example

Here, a sample dataset is used to explain nonlinear regression using SPSS software. The dataset consists of anthropometry data with the variables BMI, Height and Weight. While ‘BMI’ is dependent variable, ‘Height’ and ‘Weight’ are independent variables. The aim is to examine the impact of height and weight on BMI. The impact is measured by the regression coefficients for each independent variables. The first step is to plot the data to arrive at these initial guesses.

Table 4.1 shows the iteration history. Here, iteration history shows the progress of the clustering process at each step and also estimates the summary statistics in nonlinear regression. Therefore iteration history goes to 4 steps starting with the initial value B1 is 0, B2 is 1 and B3 is 1. This is to get a best fit result with the value of B1 equal -24.6, B2 equal 27.8 and B3 equal -.027.

MODEL PROGRAM B1=0 B2=1 B3=1.

COMPUTE PRED_= B1 + B2 * HEIGHTM + B3 * WEIGHTKG.

NLR BMI

Table 2: Nonlinear regression results of sample dataset

Table 1: Nonlinear regression results of sample dataset

Software supporting nonlinear regression

Software that support nonlinear regression test are R, SAS, MATLAB, STATA and SPSS.

Indra Giri

Indra Giri

Senior Analyst at Project Guru
He completed his Masters in Development Economics from South Asian University, New Delhi. His areas of interest includes various socio development issues like poverty, inequality and unemployment in South Asia. Apart from writing for Project Guru he loves to travel and play football in his spare time.
Indra Giri

Latest posts by Indra Giri (see all)

Related articles

  • How to conduct generalized least squares test? In statistics, Generalized Least Squares (GLS) is one of the most popular methods for estimating unknown coefficients of a linear regression model when the independent variable is correlating with the residuals.
  • How to conduct path analysis? Path analysis is a graphical representation of multiple regression models. In this analysis, the graphs represent the relationship between dependent and independent variables with the help of square and arrows.
  • How to use an instrumental variable? Instrumental variable is a third variable that estimates causal relationships in the regression analysis when an endogenous variable is present. Instrumental variables are useful when the independent variable in the regression model correlates with the error term in the model.
  • How to apply logistic regression in a case? Machine learning involves solutions to predict scenarios based on past data. Logistic regression offers probability functions based on inputs and their corresponding output.
  • How to apply linear discriminant analysis? Linear discriminant model is a multivariate model. It is used for modeling the differences in groups. In this model a categorical variable can be predicted through continuous or binary dependent variable.

Discuss

We are looking for candidates who have completed their master's degree or Ph.D. Click here to know more about our vacancies.