A Data Science Central Community

I am trying to solve the regression *Y=AX* where *Y* is the response, *X* the input, and *A* the regression coefficients. I came up with the following iterative algorithm:

*A*_{k+1} = cYU + A_{k} (I-cXU),

**where:**

**Questions:**

**Parameters:**

**Matrix dimensions:**

**Why using an iterative algorithm instead of the traditional solution?**

Note: this question is not related to the ridge regression algorithm described here.

*c*is an arbitrary constant*U*is an arbitrary matrix such that*YU*has same dimension as*A*. For instance*U*= transposed(*X*) works.*A*_{0}is the initial estimate for*A*. For instance*A*_{0}is the correlation vector between the independent variables and the response.

- What are the conditions for convergence? Do I have convergence if and only if the largest eigenvalue (in absolute value) of the matrix
*I-cXU*is strictly less than 1? - In case of convergence, will it converge to the solution of the regression problem? For instance, if
*c=0*, the algorithm converges, but not to the solution. In that case, it converges to*A*_{0}.

*n*: number of independent variables*m*: number of observations

*A*:*(1,n)*(one row,*n*columns)*I*:*(n,n)**X*:*(n,m)**U*:*(m,n)**Y*:*(1,m)*

- We are dealing with an ill-conditioned problem; most independent variables are highly correlated.
- Many solutions (as long as the regression coefficients are positive) provide a very good fit, and the global optimum is not that much better than a solution where all regression coefficients are equal to 1.
- The plan is to use an iterative algorithm to start at iteration #1 with an approximate solution that has interesting properties, then move to iteration #2 to improve a bit, then stop.

Note: this question is not related to the ridge regression algorithm described here.

**Contributions:**

- From Ray Koopman
No need to apologize for not using "proper" weights. See

Dawes, Robyn M. (1979).

*The robust beauty of improper linear models in decision making*. American Psychologist, 34, 571-582.

## You need to be a member of AnalyticBridge to add comments!

Join AnalyticBridge