本站消息

站长简介/公众号


站长简介:高级软件工程师,曾在阿里云,每日优鲜从事全栈开发工作,利用周末时间开发出本站,欢迎关注我的微信公众号:程序员总部,程序员的家,探索程序员的人生之路!分享IT最新技术,关注行业最新动向,让你永不落伍。了解同行们的工资,生活工作中的酸甜苦辣,谋求程序员的最终出路!

  价值13000svip视频教程,python大神匠心打造,零基础python开发工程师视频教程全套,基础+进阶+项目实战,包含课件和源码

  出租广告位,需要合作请联系站长

+关注
已关注

分类  

暂无分类

标签  

暂无标签

日期归档  

2021-02(23)

2021-03(37)

Python实验--线性回归+梯度下降预测波士顿房价

发布于2022-01-05 06:30     阅读(403)     评论(0)     点赞(17)     收藏(5)



1. 数据集介绍

先介绍一下将用到的数据集:
共506样本,每个样本包括13个属性以及真实房价
数据预处理:
1.从sklearn的数据库中提取boston的数据库
2.输出每个属性和房价之间的关联
3.选择关联较大的属性留下
4.划分数据集
  1. def preprocess():
  2. # get the dataset of boston
  3. X = boston().data
  4. y = boston().target
  5. name_data = boston().feature_names
  6. # draw the figure of relationship between feature and price
  7. plt.figure()
  8. for i in range(len(X[0])):
  9. plt.subplot(4, 4, i + 1)
  10. plt.scatter(X[:, i], y, s=20)
  11. plt.title(name_data[i])
  12. plt.show()
  13. # delete less relevant feature
  14. X = np.delete(X, [0, 1, 3, 4, 6, 7, 8, 9, 11], axis=1)
  15. # normalization
  16. for i in range(len(X[0])):
  17. X[:, i] = (X[:, i] - X[:, i].min()) / (X[:, i].max() - X[:, i].min())
  18. # split into test and train
  19. Xtrain, Xtest, Ytrain, Ytest = train_test_split(X, y, test_size=0.3, random_state=10)
  20. return Xtrain, Xtest, Ytrain, Ytest, X

2. 调用sklearn的线性回归函数

这个直接贴代码:

  1. def lr(Xtrain, Xtest, Ytrain, Ytest, if_figure):
  2. # use LinearRegression
  3. reg = LR().fit(Xtrain, Ytrain)
  4. y_pred = reg.predict(Xtest)
  5. loss = mean_squared_error(Ytest, y_pred)
  6. print("*************LR*****************")
  7. print("w\t= {}".format(reg.coef_))
  8. print("b\t= {:.4f}".format(reg.intercept_))
  9. # draw the figure of predict results
  10. if if_figure:
  11. plt.figure()
  12. plt.plot(range(len(Ytest)), Ytest, c="blue", label="real")
  13. plt.plot(range(len(y_pred)), y_pred, c="red", linestyle=':', label="predict")
  14. plt.title("predict results from row LR")
  15. plt.legend()
  16. plt.show()
  17. return loss

3. 手写梯度下降法

梯度下降主要思想就是以梯度作为每次迭代优化的方向,以步长更新参数,直到最优

为了两种方法比对方便,这里也使用均分误差作为Loss函数

  1. def gradDescnet(Xtrain, Xtest, Ytrain, Ytest, X, if_figure, rate):
  2. # grad descent
  3. def grad(y, yp, X):
  4. grad_w = (y - yp) * (-X)
  5. grad_b = (y - yp) * (-1)
  6. return [grad_w, grad_b]
  7. # set training parameters
  8. epoch_train = 100
  9. learning_rate = rate
  10. w = np.random.normal(0.0, 1.0, (1, len(X[0])))
  11. b = 0.0
  12. loss_train = []
  13. loss_test = []
  14. for epoch in range(epoch_train + 1):
  15. loss1 = 0
  16. for i in range(len(Xtrain)):
  17. yp = w.dot(Xtrain[i]) + b
  18. # calculate the loss
  19. err = Ytrain[i] - yp
  20. loss1 += err ** 2
  21. # iterate update w and b
  22. gw = grad(Ytrain[i], yp, Xtrain[i])[0]
  23. gb = grad(Ytrain[i], yp, Xtrain[i])[1]
  24. w = w - learning_rate * gw
  25. b = b - learning_rate * gb
  26. # record the loss
  27. loss_train.append(loss1 / len(Xtrain))
  28. loss11 = 0
  29. for i in range(len(Xtest)):
  30. yp2 = w.dot(Xtest[i]) + b
  31. err2 = Ytest[i] - yp2
  32. loss11 += err2 ** 2
  33. # record the loss
  34. loss_test.append(loss11 / len(Xtest))
  35. # shuffle the data
  36. Xtrain, Ytrain = shuffle(Xtrain, Ytrain)
  37. # draw the figure of loss
  38. if if_figure:
  39. plt.figure()
  40. plt.title("figure of loss")
  41. plt.plot(range(len(loss_train)), loss_train, c="blue", linestyle=":", label="train")
  42. plt.plot(range(len(loss_test)), loss_test, c="red", label="test")
  43. plt.legend()
  44. plt.show()
  45. # draw figure of predict results
  46. if if_figure:
  47. Predict_value = []
  48. for i in range(len(Xtest)):
  49. Predict_value.append(w.dot(Xtest[i]) + b)
  50. plt.figure()
  51. plt.title("predict results from gradScent")
  52. plt.plot(range(len(Xtest)), Ytest, c="blue", label="real")
  53. plt.plot(range(len(Xtest)), Predict_value, c="red", linestyle=':', label="predict")
  54. plt.legend()
  55. plt.show()
  56. return loss_test[-1], w, b

4. 两种方法比对

为了最终代码整洁,这里也封装为一个函数

梯度下降的步长选择0.01,这个超参数在下一部分会进行优化选择

  1. def test():
  2. if_figure = True
  3. Xtrain, Xtest, Ytrain, Ytest, X = preprocess()
  4. loss_lr = lr(Xtrain, Xtest, Ytrain, Ytest, if_figure)
  5. loss_gd, w, b = gradDescnet(Xtrain, Xtest, Ytrain, Ytest, X, if_figure, 0.01)
  6. print("*************GD*****************")
  7. print("w\t: {}".format(w))
  8. print("b\t: {}".format(b))
  9. print("************loss****************")
  10. print("lr\t: %.4f" % loss_lr)
  11. print("gd\t: %.4f" % loss_gd)

输出结果:

  1. *************LR*****************
  2. w = [ -0.39200523 21.25173835 -8.18006811 -21.61002144]
  3. b = 23.0543
  4. *************GD*****************
  5. w : [[ -0.43534889 21.65996503 -8.10720196 -21.3622824 ]]
  6. b : [22.83733711]
  7. ************loss****************
  8. lr : 31.4272
  9. gd : 31.2842

5. 超参数调整

由于迭代步长过小容易造成更新速度慢,而过长容易导致错过最优点

这里选择从0.001到0.05之间,输出步长和loss值的关系

同样封装成一个函数

  1. def searchRate():
  2. if_figure = False
  3. Xtrain, Xtest, Ytrain, Ytest, X = preprocess()
  4. loss_grad = []
  5. w_grad = []
  6. b_grad = []
  7. rates = list(np.arange(0.001, 0.05, 0.001))
  8. epoch = 1
  9. for rate in rates:
  10. loss, w, b = gradDescnet(Xtrain, Xtest, Ytrain, Ytest, X, if_figure, rate)
  11. loss_grad.append(loss[0])
  12. w_grad.append(w)
  13. b_grad.append(b)
  14. print("epoch %d: %.4f" % (epoch, loss_grad[-1]))
  15. epoch += 1
  16. plt.figure()
  17. plt.plot(rates, loss_grad)
  18. plt.title("loss under different rate")
  19. plt.show()
  20. loss_grad_min = min(loss_grad)
  21. position = loss_grad.index(loss_grad_min)
  22. w = w_grad[position]
  23. b = b_grad[position]
  24. rate = rates[position]
  25. loss_lr = lr(Xtrain, Xtest, Ytrain, Ytest, if_figure)
  26. print("*************GD*****************")
  27. print("w\t: {}".format(w))
  28. print("b\t: {}".format(b))
  29. print("rate: %.3f" % rate)
  30. print("************loss****************")
  31. print("lr\t: %.4f" % loss_lr)
  32. print("gd\t: %.4f" % loss_grad_min)

输出结果:

  1. epoch 1: 35.1047
  2. epoch 2: 31.9512
  3. epoch 3: 31.6400
  4. epoch 4: 31.8814
  5. epoch 5: 31.3429
  6. epoch 6: 31.7260
  7. epoch 7: 31.5825
  8. epoch 8: 31.5523
  9. epoch 9: 32.4876
  10. epoch 10: 31.4287
  11. epoch 11: 31.1475
  12. epoch 12: 32.0841
  13. epoch 13: 32.0033
  14. epoch 14: 31.5768
  15. epoch 15: 31.1828
  16. epoch 16: 31.6558
  17. epoch 17: 32.2582
  18. epoch 18: 32.4916
  19. epoch 19: 31.2118
  20. epoch 20: 32.2877
  21. epoch 21: 31.7237
  22. epoch 22: 32.1203
  23. epoch 23: 32.7307
  24. epoch 24: 32.7434
  25. epoch 25: 32.6421
  26. epoch 26: 31.8588
  27. epoch 27: 31.1762
  28. epoch 28: 33.0360
  29. epoch 29: 32.5580
  30. epoch 30: 32.4591
  31. epoch 31: 31.4191
  32. epoch 32: 31.1398
  33. epoch 33: 31.4291
  34. epoch 34: 31.3900
  35. epoch 35: 31.2239
  36. epoch 36: 31.4200
  37. epoch 37: 31.2967
  38. epoch 38: 32.5322
  39. epoch 39: 32.3174
  40. epoch 40: 34.3984
  41. epoch 41: 31.1794
  42. epoch 42: 31.8992
  43. epoch 43: 32.0060
  44. epoch 44: 34.0944
  45. epoch 45: 34.3244
  46. epoch 46: 31.1479
  47. epoch 47: 32.8374
  48. epoch 48: 31.7111
  49. epoch 49: 33.6676
  50. *************LR*****************
  51. w = [ -0.39200523 21.25173835 -8.18006811 -21.61002144]
  52. b = 23.0543
  53. *************GD*****************
  54. w : [[ -0.29030409 21.60092767 -8.02647596 -21.79164094]]
  55. b : [23.35049725]
  56. rate: 0.032
  57. ************loss****************
  58. lr : 31.4272
  59. gd : 31.1398

可见调整步长对于最终结果还是有较大影响的

6. 导入库汇总

  1. from sklearn.linear_model import LinearRegression as LR
  2. from sklearn.model_selection import train_test_split
  3. from sklearn.datasets import load_boston as boston
  4. import matplotlib.pyplot as plt
  5. from sklearn.utils import shuffle
  6. import numpy as np
  7. from sklearn.metrics import mean_squared_error

原文链接:https://blog.csdn.net/weixin_43909400/article/details/122296234







所属网站分类: 技术文章 > 博客

作者:小鬼来了

链接:https://www.pythonheidong.com/blog/article/1219620/8404d88806508fd0e1a1/

来源:python黑洞网

任何形式的转载都请注明出处,如有侵权 一经发现 必将追究其法律责任

17 0
收藏该文
已收藏

评论内容:(最多支持255个字符)