Logistic回归

Halcom

Logistic回归
参考Softmax回归，

以数字图像为例进行研究，X为500x400的矩阵，500表示500张图像，每个图像20x20；y为500x1的矩阵，标签为1:10；主函数为：

1. clc,clear,close all
   
2. load('Xy.mat')
   
3. % X为500x400的矩阵，500表示500张图像，每个图像20x20；
   
4. % y为500x1的矩阵，标签为1:10；
   
5. lambda = 0.1;  % lambda为正则化参数抑制过拟合
   
6. [all_theta] = oneVsAll(X, y, num_labels, lambda);
   
7. 
   
8. % 预测
   
9. % pred = predictOneVsAll(all_theta, X);
   
10. m = size(X, 1);   % m = 500
    
11. num_labels = size(all_theta, 1);   % num_labels = 10
    
12. pred = zeros(size(X, 1), 1);
    
13. % Add ones to the X data matrix
    
14. X = [ones(m, 1) X];
    
15. [k, pred] = max(X * all_theta' , [], 2);

复制代码

多分类优化theta，

1. function [all_theta] = oneVsAll(X, y, num_labels, lambda)
   
2. % 多类分类回归
   
3. % 当训练数字1时，数字1为一类1，则其它样本数字均为另一类0
   
4. % 当训练数字2时，数字2为一类1，则其它样本数字均为另一类0
   
5. % 当训练数字3时，数字3为一类1，则其它样本数字均为另一类0
   
6. % ……
   
7. % 当训练数字10时，10表示0，数字10为一类1，则其它样本数字均为另一类0
   
8. 
   
9. m = size(X, 1);  % m = 500 样本数
   
10. n = size(X, 2);  % n = 400 特征数
    
11. 
    
12. % 初始化 10x401
    
13. all_theta = zeros(num_labels, n + 1);
    
14. 
    
15. % 增加直流分量
    
16. X = [ones(m, 1), X];
    
17. 
    
18. % Set Initial theta
    
19. initial_theta = zeros(n + 1, 1);
    
20. %
    
21. % % Set options for fminunc
    
22. options = optimset('GradObj', 'on', 'MaxIter', 50);
    
23. %
    
24. %  % Run fmincg to obtain the optimal theta
    
25. %  % This function will return theta and the cost
    
26. for iter = 1:num_labels
    
27.     % J最小时, theta的取值
    
28.     all_theta(iter,:)= fmincg (@(t)(lrCostFunction(t, X, (y == iter), lambda)), initial_theta, options);
    
29. end
    

复制代码

Logistic回归损失函数如下：

1. function [J, grad] = lrCostFunction(theta, X, y, lambda)
   
2. % Compute cost and gradient for logistic regression with regularization
   
3. 
   
4. % Initialize some useful values
   
5. m = length(y); % number of training examples
   
6. 
   
7. % You need to return the following variables correctly
   
8. J = 0;
   
9. grad = zeros(size(theta));
   
10. %  grad = grad + YOUR_CODE_HERE (using the temp variable)
    
11. 
    
12. h = sigmoid(X*theta);
    
13. A = theta(2:size(theta),1)
    
14. J = ((-y)'*log(h)-(1-y)'*log(1-h))/m + lambda/(2*m)*sum(A.^2);
    
15. 
    
16. % calculate grads
    
17. grad = (X'*(h - y))/m + lambda*(1/m)*theta;
    
18. grad(1)= grad(1) - lambda*(1/m)*theta(1);
    
19. % ========================================
    
20. grad = grad(:);
    
21. 
    
22. end

复制代码

参考：
【1】Softmax回归