sklearn包MLPClassifier的使用详解+例子

MLPClassifier

参数说明

1. hidden_layer_sizes : 元组形式,长度n_layers-2,默认(100,),第i元素表示第i个神经元的个数
2. activation: {‘identity’, ‘logistic’, ‘tanh’, ‘relu’},默认"relu"

• ‘identity’: f(x) = x
• ‘logistic’:f(x) = 1 / (1 + exp(-x))
• ‘tanh’ : f(x) = tanh(x)
• ‘relu’ : f(x) = max(0, x)

1. solver:{‘lbfgs’, ‘sgd’, ‘adam’}, default ‘adam’

• lbfgs：quasi-Newton方法的优化器
• sgd：随机梯度下降
• adam： Kingma, Diederik, and Jimmy Ba提出的机遇随机梯度的优化器 注意：默认solver ‘adam’在相对较大的数据集上效果比较好（几千个样本或者更多），对小数据集来说，lbfgs收敛更快效果也更好

1. alpha:float,可选的，默认0.0001,正则化项参数
2. batch_size:int , 可选的，默认‘auto’,随机优化的minibatches的大小，如果solver是‘lbfgs’，分类器将不使用minibatch，当设置成‘auto’，batch_size=min(200,n_samples)
3. learning_rate:{‘constant’，‘invscaling’, ‘adaptive’},默认‘constant’，用于权重更新，只有当solver为’sgd‘时使用

• ‘constant’: 有‘learning_rate_init’给定的恒定学习率
• ‘incscaling’：随着时间t使用’power_t’的逆标度指数不断降低学习率learning_rate_effective_learning_rate = learning_rate_init / pow(t, power_t)
• ‘adaptive’：只要训练损耗在下降，就保持学习为’learning_rate_init’不变，当连续两次不能降低训练损耗或验证分数停止升高至少tol时，将当前学习率除以5.

1. max_iter: int，可选，默认200，最大迭代次数。
2. random_state:int 或RandomState，可选，默认None，随机数生成器的状态或种子
3. shuffle: bool，可选，默认True,只有当solver=’sgd’或者‘adam’时使用，判断是否在每次迭代时对样本进行清洗。
4. tol：float, 可选，默认1e-4，优化的容忍度
5. learning_rate_int:double,可选，默认0.001，初始学习率，控制更新权重的补偿，只有当solver=’sgd’ 或’adam’时使用。
6. power_t: double, optional, default 0.5，只有solver=’sgd’时使用，是逆扩展学习率的指数.当learning_rate=’invscaling’，用来更新有效学习率。
7. verbose : bool, optional, default False,是否将过程打印到stdout
8. warm_start : bool, optional, default False,当设置成True，使用之前的解决方法作为初始拟合，否则释放之前的解决方法
9. momentum : float, default 0.9,Momentum(动量） for gradient descent update. Should be between 0 and 1. Only used when solver=’sgd’.
10. nesterovs_momentum : boolean, default True, Whether to use Nesterov’s momentum. Only used when solver=’sgd’ and momentum > 0
11. early_stopping : bool, default False,Only effective when solver=’sgd’ or ‘adam’,判断当验证效果不再改善的时候是否终止训练，当为True时，自动选出10%的训练数据用于验证并在两步连续爹迭代改善低于tol时终止训练
12. validation_fraction: float, optional, default 0.1,用作早期停止验证的预留训练数据集的比例，早0-1之间，只当early_stopping=True有用
13. beta_1 : float, optional, default 0.9，Only used when solver=’adam’，估计一阶矩向量的指数衰减速率，[0,1)之间
14. beta_2 : float, optional, default 0.999,Only used when solver=’adam’估计二阶矩向量的指数衰减速率[0,1)之间
15. psilon: float, optional, default 1e-8,Only used when solver=’adam’数值稳定值。

属性说明

1. classes_:每个输出的类标签
2. loss_:损失函数计算出来的当前损失值
3. coefs_:列表中的第i个元素表示i层的权重矩阵
4. intercepts_:列表中第i个元素代表i+1层的偏差向量
5. n_iter_：迭代次数
6. n_layers_:层数
7. n_outputs_:输出的个数
8. out_activation_:输出激活函数的名称。

方法说明

• fit(X,y):拟合
• get_params([deep]):获取参数
• predict(X):使用MLP进行预测
• predic_log_proba(X):返回对数概率估计
• predic_proba(X)：概率估计
• score(X,y[,sample_weight]):返回给定测试数据和标签上的平均准确度
• set_params(**params):设置参数。

from sklearn.neural_network import MLPClassifier
X = [[0., 0.], [1., 1.]]
y = [0, 1]
clf = MLPClassifier(solver='lbfgs', alpha=1e-5,
                    hidden_layer_sizes=(5, 2), random_state=1)
clf.fit(X, y)
MLPClassifier(activation='relu', alpha=1e-05, batch_size='auto', beta_1=0.9,
       beta_2=0.999, early_stopping=False, epsilon=1e-08,
       hidden_layer_sizes=(5, 2), learning_rate='constant',
       learning_rate_init=0.001, max_iter=200, momentum=0.9,
       nesterovs_momentum=True, power_t=0.5, random_state=1, shuffle=True,
       solver='lbfgs', tol=0.0001, validation_fraction=0.1, verbose=False,
       warm_start=False)
print(clf.predict([[2., 2.], [-1., -2.]]))
# array([1, 0])
print(clf.predict_proba([[2., 2.], [-1.,- 2.]]))
# array([[  1.96718015e-004,   9.99803282e-001],
        #[  1.00000000e+000,   4.67017947e-144]])

例子一：iris三分类器

from sklearn.neural_network import MLPClassifier

from BP_Net import normalized, load_csv
import numpy as np

if __name__ == '__main__':
    X, Y = load_csv()
    X = normalized(X)
    Y[np.where(Y == 1)] = 2
    Y[np.where(Y == 0.5)] = 1
    # Y = normalized(Y)
    """训练集90个数据"""
    train_x = np.hstack((X[:, 0:30], X[:, 50:80], X[:, 100:130]))
    train_x = train_x.T
    train_y = np.hstack((Y[:, 0:30], Y[:, 50:80], Y[:, 100:130]))
    """测试集60个数据"""
    test_x = np.hstack((X[:, 30:50], X[:, 80:100], X[:, 130:150]))
    test_x = test_x.T
    test_y = np.hstack((Y[:, 30:50], Y[:, 80:100], Y[:, 130:150]))
    # 首先，创建一个多分类模型对象 类似于Java的类调用
    # 括号中填写多个参数，如果不写，则使用默认值，我们一般要构建隐层结构，调试正则化参数，设置最大迭代次数
    mlp = MLPClassifier(hidden_layer_sizes=(10,), alpha=0.01, max_iter=10000)
    # 调用fit函数就可以进行模型训练，一般的调用模型函数的训练方法都是fit()
    # print(train_x.shape)
    # print(train_y.ravel().shape)
    mlp.fit(train_x, train_y.ravel())  # 这里y值需要注意，还原成一维数组
    # 模型就这样训练好了，而后我们可以调用多种函数来获取训练好的参数
    # 比如获取准确率
    print('训练集的准确率是：', mlp.score(test_x, test_y.ravel()))
    # 比如输出当前的代价值
    print('训练集的代价值是：', mlp.loss_)
    # 比如输出每个theta的权重
    print('训练集的权重值是：', mlp.coefs_)

结果

训练集的准确率是： 1.0
训练集的代价值是： 0.05801992526160142
训练集的权重值是： [array([[ 1.31733465e+00, -2.93551147e-01,  1.03982794e+00,
        -3.04550107e-04, -8.51421595e-01,  6.62205673e-29,
         2.11835613e-15,  8.08139861e-01, -1.23505242e-06,
        -5.51082075e-01],
       [-5.79154193e-01,  1.36001217e+00, -1.12722611e+00,
         1.33249760e-15,  9.80136514e-01, -2.05913410e-02,
        -1.15404725e-02, -6.38918400e-01,  4.31038716e-05,
        -2.21295744e-01],
       [ 1.35044205e+00, -1.04249680e+00,  8.52670431e-01,
        -8.03660618e-28, -1.06280029e+00,  6.88455552e-26,
        -1.49624759e-06,  1.17238104e+00,  6.78174401e-05,
        -1.43132703e-02],
       [ 1.64364171e+00, -9.59131062e-01,  1.31785190e+00,
        -3.96164651e-05, -9.56257701e-01, -1.44515932e-02,
         3.95722198e-28,  6.42796159e-01, -1.20803943e-05,
        -2.97102291e-04]], dtype=float32), array([[ 1.0823956e+00],
       [-1.6924484e+00],
       [ 1.4192194e+00],
       [-3.2202732e-02],
       [-1.4481077e+00],
       [-1.8761203e-03],
       [ 1.9695616e-04],
       [ 1.6395217e+00],
       [-4.0499594e-02],
       [ 7.0304796e-02]], dtype=float32)]

例子二：手写数字识别

from sklearn.neural_network import MLPClassifier
# from sklearn.datasets import fetch_mldata
import numpy as np
import pickle
import gzip

# 加载数据
# mnist = fetch_mldata("MNIST original")
with gzip.open("mnist.pkl.gz") as fp:
    training_data, valid_data, test_data = pickle.load(fp, encoding='bytes')
x_training_data, y_training_data = training_data
x_valid_data, y_valid_data = valid_data
x_test_data, y_test_data = test_data
classes = np.unique(y_test_data)

# 将验证集和训练集合并
x_training_data_final = np.vstack((x_training_data, x_valid_data))
y_training_data_final = np.append(y_training_data, y_valid_data)

# 设置神经网络模型参数
# mlp = MLPClassifier(solver='lbfgs', activation='relu',alpha=1e-4,hidden_layer_sizes=(50,50), random_state=1,max_iter=10,verbose=10,learning_rate_init=.1)
# 使用solver='lbfgs',准确率为79%，比较适合小(少于几千)数据集来说，且使用的是全训练集训练，比较消耗内存
# mlp = MLPClassifier(solver='adam', activation='relu',alpha=1e-4,hidden_layer_sizes=(50,50), random_state=1,max_iter=10,verbose=10,learning_rate_init=.1)
# 使用solver='adam'，准确率只有67%
mlp = MLPClassifier(solver='sgd', activation='relu', alpha=1e-4, hidden_layer_sizes=(50, 50), random_state=1,
                    max_iter=100, verbose=True, learning_rate_init=.1)
# 使用solver='sgd'，准确率为98%，且每次训练都会分batch，消耗更小的内存

# 训练模型
mlp.fit(x_training_data_final, y_training_data_final)

# 查看模型结果
print(mlp.score(x_test_data, y_test_data))
print(mlp.n_layers_)
print(mlp.n_iter_)
print(mlp.loss_)
print(mlp.out_activation_)

结果

Iteration 1, loss = 0.31443422
Iteration 2, loss = 0.13076474
Iteration 3, loss = 0.09742518
Iteration 4, loss = 0.08100330
Iteration 5, loss = 0.06801912
Iteration 6, loss = 0.06218105
Iteration 7, loss = 0.05417376
Iteration 8, loss = 0.04865488
Iteration 9, loss = 0.04225277
Iteration 10, loss = 0.03999533
Iteration 11, loss = 0.03581450
Iteration 12, loss = 0.03553377
Iteration 13, loss = 0.02851309
Iteration 14, loss = 0.02561775
Iteration 15, loss = 0.02522932
Iteration 16, loss = 0.02467297
Iteration 17, loss = 0.02161946
Iteration 18, loss = 0.02143663
Iteration 19, loss = 0.02414556
Iteration 20, loss = 0.02093072
Iteration 21, loss = 0.02043619
Iteration 22, loss = 0.02022548
Iteration 23, loss = 0.01801227
Iteration 24, loss = 0.01937727
Iteration 25, loss = 0.02075462
Iteration 26, loss = 0.01892496
Iteration 27, loss = 0.01754461
Iteration 28, loss = 0.01478817
Iteration 29, loss = 0.01456456
Iteration 30, loss = 0.01663158
Iteration 31, loss = 0.01425532
Iteration 32, loss = 0.01702378
Iteration 33, loss = 0.01619255
Iteration 34, loss = 0.01835025
Iteration 35, loss = 0.01920801
Iteration 36, loss = 0.01692277
Iteration 37, loss = 0.01762001
Iteration 38, loss = 0.01061955
Iteration 39, loss = 0.01233185
Iteration 40, loss = 0.01695161
Iteration 41, loss = 0.01152016
Iteration 42, loss = 0.01516701
Iteration 43, loss = 0.02044881
Iteration 44, loss = 0.01657610
Iteration 45, loss = 0.01598418
Iteration 46, loss = 0.01809941
Iteration 47, loss = 0.02124500
Iteration 48, loss = 0.01304665
Iteration 49, loss = 0.00769560
Iteration 50, loss = 0.01211744
Iteration 51, loss = 0.01232973
Iteration 52, loss = 0.01403902
Iteration 53, loss = 0.01696003
Iteration 54, loss = 0.01544194
Iteration 55, loss = 0.01286083
Iteration 56, loss = 0.01341688
Iteration 57, loss = 0.00990281
Iteration 58, loss = 0.00681194
Iteration 59, loss = 0.01107953
Iteration 60, loss = 0.01781647
Iteration 61, loss = 0.01664884
Iteration 62, loss = 0.01788967
Iteration 63, loss = 0.02161008
Iteration 64, loss = 0.01799941
Iteration 65, loss = 0.02034798
Iteration 66, loss = 0.01451698
Iteration 67, loss = 0.01982461
Iteration 68, loss = 0.01926753
Iteration 69, loss = 0.01591123
Training loss did not improve more than tol=0.000100 for 10 consecutive epochs. Stopping.
0.972
4
69
0.0159112293736664
softmax