Scikit-learn 不允许对系数进行此类限制。
但是您可以对系数施加任何限制并使用以下方法优化损失坐标下降 if you 实现你自己的估算器。在无约束的情况下,坐标下降在合理的迭代次数下产生与 OLS 相同的结果。
我编写了一个类,对线性回归系数施加上限和下限。如果您愿意,您可以将其扩展为使用 Ridge 或 evel Lasso 惩罚:
from sklearn.linear_model.base import LinearModel
from sklearn.base import RegressorMixin
from sklearn.utils import check_X_y
import numpy as np
class ConstrainedLinearRegression(LinearModel, RegressorMixin):
def __init__(self, fit_intercept=True, normalize=False, copy_X=True, nonnegative=False, tol=1e-15):
self.fit_intercept = fit_intercept
self.normalize = normalize
self.copy_X = copy_X
self.nonnegative = nonnegative
self.tol = tol
def fit(self, X, y, min_coef=None, max_coef=None):
X, y = check_X_y(X, y, accept_sparse=['csr', 'csc', 'coo'], y_numeric=True, multi_output=False)
X, y, X_offset, y_offset, X_scale = self._preprocess_data(
X, y, fit_intercept=self.fit_intercept, normalize=self.normalize, copy=self.copy_X)
self.min_coef_ = min_coef if min_coef is not None else np.repeat(-np.inf, X.shape[1])
self.max_coef_ = max_coef if max_coef is not None else np.repeat(np.inf, X.shape[1])
if self.nonnegative:
self.min_coef_ = np.clip(self.min_coef_, 0, None)
beta = np.zeros(X.shape[1]).astype(float)
prev_beta = beta + 1
hessian = np.dot(X.transpose(), X)
while not (np.abs(prev_beta - beta)<self.tol).all():
prev_beta = beta.copy()
for i in range(len(beta)):
grad = np.dot(np.dot(X,beta) - y, X)
beta[i] = np.minimum(self.max_coef_[i],
np.maximum(self.min_coef_[i],
beta[i]-grad[i] / hessian[i,i]))
self.coef_ = beta
self._set_intercept(X_offset, y_offset, X_scale)
return self
例如,您可以使用此类使所有系数非负
from sklearn.datasets import load_boston
from sklearn.linear_model import LinearRegression
X, y = load_boston(return_X_y=True)
model = ConstrainedLinearRegression(nonnegative=True)
model.fit(X, y)
print(model.intercept_)
print(model.coef_)
这会产生类似的输出
-36.99292986145538
[0. 0.05286515 0. 4.12512386 0. 8.04017956
0. 0. 0. 0. 0. 0.02273805
0. ]
您可以看到大多数系数为零。普通的 LinearModel 会使它们变为负值:
model = LinearRegression()
model.fit(X, y)
print(model.intercept_)
print(model.coef_)
它将返回给你
36.49110328036191
[-1.07170557e-01 4.63952195e-02 2.08602395e-02 2.68856140e+00
-1.77957587e+01 3.80475246e+00 7.51061703e-04 -1.47575880e+00
3.05655038e-01 -1.23293463e-02 -9.53463555e-01 9.39251272e-03
-5.25466633e-01]
您还可以为您选择的任何系数施加任意范围 - 这就是您所要求的。例如,在此设置中
model = ConstrainedLinearRegression()
min_coef = np.repeat(-np.inf, X.shape[1])
min_coef[0] = 0
min_coef[4] = -1
max_coef = np.repeat(4, X.shape[1])
max_coef[3] = 2
model.fit(X, y, max_coef=max_coef, min_coef=min_coef)
print(model.intercept_)
print(model.coef_)
你会得到一个输出
24.060175576410515
[ 0. 0.04504673 -0.0354073 2. -1. 4.
-0.01343263 -1.17231216 0.2183103 -0.01375266 -0.7747823 0.01122374
-0.56678676]
Update。该解决方案可以适应对系数线性组合(例如它们的总和)的约束 - 在这种情况下,每个系数的单独约束将在每个步骤中重新计算。这个 Github 要点 http://gist.github.com/avidale/6668635c318aceebe0142de013a4cf77提供了一个例子。
UPDATE由于这个问题很受欢迎,我创建了一个包含约束线性回归实现的包:https://github.com/avidale/constrained-线性-回归 https://github.com/avidale/constrained-linear-regression。
你可以安装它pip install constrained-linear-regression
。欢迎请求请求!