Abstract: In this paper, a vector parameter method for ridge regression is proposed. We choose the negative gradient of mean square error as vector direction and decide vector norm with the expectation constrains both of mean square error and of residual error. We come to conclusions that the mean square error is a decreasing function of vector norm while the residual error a increasing one. It is the monotonicity of the errors that leads to our expectation constrains. Since two conflict constrains are under consideration, our vector parameter ridge regression is expected to bear both satisfactory mean square error and acceptable residual error. Finally, a multi-collinearity model is given as an example.