深度学习算法_吴恩达指导_深度学习_人工智能/神经网络/深度学习

<html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta charset="utf-8"> <meta name="generator" content="pdf2htmlEX"> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <link rel="stylesheet" href="https://static.pudn.com/base/css/base.min.css"> <link rel="stylesheet" href="https://static.pudn.com/base/css/fancy.min.css"> <link rel="stylesheet" href="https://static.pudn.com/prod/directory_preview_static/624704d962b5053d3c7dc90f/raw.css"> <script src="https://static.pudn.com/base/js/compatibility.min.js"></script> <script src="https://static.pudn.com/base/js/pdf2htmlEX.min.js"></script> <script> try{ pdf2htmlEX.defaultViewer = new pdf2htmlEX.Viewer({}); }catch(e){} </script> <title></title> </head> <body> <div id="sidebar" style="display: none"> <div id="outline"> </div> </div> <div id="pf1" class="pf w0 h0" data-page-no="1"><div class="pc pc1 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/624704d962b5053d3c7dc90f/bg1.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">DRAFT</div><div class="t m1 x2 h3 y2 ff2 fs1 fc1 sc0 ls0 ws0">Contents</div><div class="t m1 x2 h4 y3 ff3 fs2 fc1 sc0 ls0 ws0">1 前言 1</div><div class="t m1 x3 h5 y4 ff5 fs2 fc1 sc0 ls0 ws0">1.1 谁应该读这本书？ . . . . . . . . . . . . . . . . . . . . . . . . . . . 9</div><div class="t m1 x3 h5 y5 ff5 fs2 fc1 sc0 ls0 ws0">1.2 深度学习的历史趋势 . . . . . . . . . . . . . . . . . . . . . . . . . 11</div><div class="t m1 x4 h5 y6 ff5 fs2 fc1 sc0 ls0 ws0">1.2.1 神经网络的众多名称和命运变迁 . . . . . . . . . . . . . . . 11</div><div class="t m1 x4 h5 y7 ff5 fs2 fc1 sc0 ls0 ws0">1.2.2 与日俱增的数据量 . . . . . . . . . . . . . . . . . . . . . . . 15</div><div class="t m1 x4 h5 y8 ff5 fs2 fc1 sc0 ls0 ws0">1.2.3 与日俱增的模型规模 . . . . . . . . . . . . . . . . . . . . . 16</div><div class="t m1 x4 h5 y9 ff5 fs2 fc1 sc0 ls0 ws0">1.2.4 与日俱增的精度、复杂度和对现实世界的冲击 . . . . . . . 17</div><div class="t m1 x2 h4 ya ff3 fs2 fc1 sc0 ls0 ws0">2 线性代数 21</div><div class="t m1 x3 h5 yb ff5 fs2 fc1 sc0 ls0 ws0">2.1 标量，向量，矩阵和张量 . . . . . . . . . . . . . . . . . . . . . . . 21</div><div class="t m1 x3 h5 yc ff5 fs2 fc1 sc0 ls0 ws0">2.2 矩阵和向量相乘 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23</div><div class="t m1 x3 h5 yd ff5 fs2 fc1 sc0 ls0 ws0">2.3 单位矩阵和逆矩阵 . . . . . . . . . . . . . . . . . . . . . . . . . . . 25</div><div class="t m1 x3 h5 ye ff5 fs2 fc1 sc0 ls0 ws0">2.4 线性相关和生成子空间 . . . . . . . . . . . . . . . . . . . . . . . . 26</div><div class="t m1 x3 h5 yf ff5 fs2 fc1 sc0 ls0 ws0">2.5 范数 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27</div><div class="t m1 x3 h5 y10 ff5 fs2 fc1 sc0 ls0 ws0">2.6 特殊类型的矩阵和向量 . . . . . . . . . . . . . . . . . . . . . . . . 29</div><div class="t m1 x3 h5 y11 ff5 fs2 fc1 sc0 ls0 ws0">2.7 特征分解 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30</div><div class="t m1 x3 h5 y12 ff5 fs2 fc1 sc0 ls0 ws0">2.8 奇异值分解 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32</div><div class="t m1 x3 h5 y13 ff5 fs2 fc1 sc0 ls0 ws0">2.9 Moore-Penrose 伪逆 . . . . . . . . . . . . . . . . . . . . . . . . . . 33</div><div class="t m1 x3 h5 y14 ff5 fs2 fc1 sc0 ls0 ws0">2.10 迹运算 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34</div><div class="t m1 x3 h5 y15 ff5 fs2 fc1 sc0 ls0 ws0">2.11 行列式 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34</div><div class="t m1 x3 h5 y16 ff5 fs2 fc1 sc0 ls0 ws0">2.12 实例：主成分分析 . . . . . . . . . . . . . . . . . . . . . . . . . . . 35</div><div class="t m1 x2 h4 y17 ff3 fs2 fc1 sc0 ls0 ws0">3 概率与信息论 39</div><div class="t m1 x3 h5 y18 ff5 fs2 fc1 sc0 ls0 ws0">3.1 为什么要用概率？ . . . . . . . . . . . . . . . . . . . . . . . . . . . 39</div><div class="t m1 x3 h5 y19 ff5 fs2 fc1 sc0 ls0 ws0">3.2 随机变量 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41</div><div class="t m1 x3 h5 y1a ff5 fs2 fc1 sc0 ls0 ws0">3.3 概率分布 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41</div><div class="t m1 x4 h5 y1b ff5 fs2 fc1 sc0 ls0 ws0">3.3.1 离散型变量和概率分布律函数 . . . . . . . . . . . . . . . . 42</div><div class="t m1 x4 h5 y1c ff5 fs2 fc1 sc0 ls0 ws0">3.3.2 连续型变量和概率密度函数 . . . . . . . . . . . . . . . . . . 43</div><div class="t m1 x3 h5 y1d ff5 fs2 fc1 sc0 ls0 ws0">3.4 边缘概率 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43</div><div class="t m1 x5 h5 y1e ff5 fs2 fc1 sc0 ls0 ws0">1</div><div class="c x6 y1f w2 h6"><div class="t m1 x0 h7 y20 ff6 fs3 fc1 sc0 ls0 ws0">深度学习算法</div></div><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a></div><div class="pi" data-data='{"ctm":[1.904762,0.000000,0.000000,1.904762,0.000000,0.000000]}'></div></div> </body> </html>

<div id="pf2" class="pf w0 h0" data-page-no="2"><div class="pc pc2 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/624704d962b5053d3c7dc90f/bg2.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">DRAFT</div><div class="t m1 x2 h5 y21 ff7 fs2 fc1 sc0 ls0 ws0">CONTENTS 2</div><div class="t m1 x3 h5 y22 ff5 fs2 fc1 sc0 ls0 ws0">3.5 条件概率 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44</div><div class="t m1 x3 h5 y23 ff5 fs2 fc1 sc0 ls0 ws0">3.6 条件概率的链式法则 . . . . . . . . . . . . . . . . . . . . . . . . . 44</div><div class="t m1 x3 h5 y24 ff5 fs2 fc1 sc0 ls0 ws0">3.7 独立性和条件独立性 . . . . . . . . . . . . . . . . . . . . . . . . . 44</div><div class="t m1 x3 h5 y25 ff5 fs2 fc1 sc0 ls0 ws0">3.8 期望，方差和协方差 . . . . . . . . . . . . . . . . . . . . . . . . . 45</div><div class="t m1 x3 h5 y26 ff5 fs2 fc1 sc0 ls0 ws0">3.9 常用概率分布 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46</div><div class="t m1 x4 h5 y27 ff5 fs2 fc1 sc0 ls0 ws0">3.9.1 Bernoulli 分布 . . . . . . . . . . . . . . . . . . . . . . . . . 47</div><div class="t m1 x4 h5 y28 ff5 fs2 fc1 sc0 ls0 ws0">3.9.2 multinoulli 分布 . . . . . . . . . . . . . . . . . . . . . . . . 47</div><div class="t m1 x4 h5 y29 ff5 fs2 fc1 sc0 ls0 ws0">3.9.3 高斯分布 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47</div><div class="t m1 x4 h5 y2a ff5 fs2 fc1 sc0 ls0 ws0">3.9.4 指数分布和Laplace 分布 . . . . . . . . . . . . . . . . . . . 49</div><div class="t m1 x4 h5 y2b ff5 fs2 fc1 sc0 ls0 ws0">3.9.5 Dirac 分布和经验分布 . . . . . . . . . . . . . . . . . . . . . 49</div><div class="t m1 x4 h5 y2c ff5 fs2 fc1 sc0 ls0 ws0">3.9.6 分布的混合 . . . . . . . . . . . . . . . . . . . . . . . . . . . 50</div><div class="t m1 x3 h5 y2d ff5 fs2 fc1 sc0 ls0 ws0">3.10 常用函数的一些性质 . . . . . . . . . . . . . . . . . . . . . . . . . 51</div><div class="t m1 x3 h5 y2e ff5 fs2 fc1 sc0 ls0 ws0">3.11 贝叶斯规则 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53</div><div class="t m1 x3 h5 y2f ff5 fs2 fc1 sc0 ls0 ws0">3.12 连续型变量的技术细节 . . . . . . . . . . . . . . . . . . . . . . . . 54</div><div class="t m1 x3 h5 y30 ff5 fs2 fc1 sc0 ls0 ws0">3.13 信息论 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55</div><div class="t m1 x3 h5 y31 ff5 fs2 fc1 sc0 ls0 ws0">3.14 结构化概率模型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58</div><div class="t m1 x2 h4 y32 ff3 fs2 fc1 sc0 ls0 ws0">4 数值计算 61</div><div class="t m1 x3 h5 y33 ff5 fs2 fc1 sc0 ls0 ws0">4.1 上溢和下溢 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61</div><div class="t m1 x3 h5 y34 ff5 fs2 fc1 sc0 ls0 ws0">4.2 病态条件数 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62</div><div class="t m1 x3 h5 y35 ff5 fs2 fc1 sc0 ls0 ws0">4.3 基于梯度的优化方法 . . . . . . . . . . . . . . . . . . . . . . . . . 63</div><div class="t m1 x4 h5 y36 ff5 fs2 fc1 sc0 ls0 ws0">4.3.1 梯度之上：雅各比和海森矩阵 . . . . . . . . . . . . . . . . 66</div><div class="t m1 x3 h5 y37 ff5 fs2 fc1 sc0 ls0 ws0">4.4 约束优化 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70</div><div class="t m1 x3 h5 y38 ff5 fs2 fc1 sc0 ls0 ws0">4.5 实例：线性最小二乘 . . . . . . . . . . . . . . . . . . . . . . . . . 72</div><div class="t m1 x2 h4 y39 ff3 fs2 fc1 sc0 ls0 ws0">5 机器学习基础 74</div><div class="t m1 x3 h5 y3a ff5 fs2 fc1 sc0 ls0 ws0">5.1 学习算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74</div><div class="t m1 x4 h5 y3b ff5 fs2 fc1 sc0 ls0 ws0">5.1.1 任务，T . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75</div><div class="t m1 x4 h5 y3c ff5 fs2 fc1 sc0 ls0 ws0">5.1.2 性能衡量，P . . . . . . . . . . . . . . . . . . . . . . . . . . 77</div><div class="t m1 x4 h5 y3d ff5 fs2 fc1 sc0 ls0 ws0">5.1.3 经验，E . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78</div><div class="t m1 x4 h5 y3e ff5 fs2 fc1 sc0 ls0 ws0">5.1.4 实例：线性回归 . . . . . . . . . . . . . . . . . . . . . . . . 80</div><div class="t m1 x3 h5 y3f ff5 fs2 fc1 sc0 ls0 ws0">5.2 容量，过拟合和欠拟合 . . . . . . . . . . . . . . . . . . . . . . . . 83</div><div class="t m1 x4 h5 y40 ff5 fs2 fc1 sc0 ls0 ws0">5.2.1 没有免费午餐定理 . . . . . . . . . . . . . . . . . . . . . . . 87</div><div class="t m1 x4 h5 y41 ff5 fs2 fc1 sc0 ls0 ws0">5.2.2 正则化 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88</div><div class="t m1 x3 h5 y42 ff5 fs2 fc1 sc0 ls0 ws0">5.3 超参数和验证集 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90</div><div class="t m1 x4 h5 y43 ff5 fs2 fc1 sc0 ls0 ws0">5.3.1 交叉验证 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91</div><div class="t m1 x3 h5 y44 ff5 fs2 fc1 sc0 ls0 ws0">5.4 估计，偏差和方差 . . . . . . . . . . . . . . . . . . . . . . . . . . . 91</div><div class="t m1 x4 h5 y45 ff5 fs2 fc1 sc0 ls0 ws0">5.4.1 点估计 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91</div><div class="t m1 x4 h5 y46 ff5 fs2 fc1 sc0 ls0 ws0">5.4.2 偏差 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92</div><div class="t m1 x4 h5 y47 ff5 fs2 fc1 sc0 ls0 ws0">5.4.3 方差和标准差 . . . . . . . . . . . . . . . . . . . . . . . . . 95</div><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a></div><div class="pi" data-data='{"ctm":[1.904762,0.000000,0.000000,1.904762,0.000000,0.000000]}'></div></div>

<div id="pf3" class="pf w0 h0" data-page-no="3"><div class="pc pc3 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/624704d962b5053d3c7dc90f/bg3.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">DRAFT</div><div class="t m1 x2 h5 y21 ff7 fs2 fc1 sc0 ls0 ws0">CONTENTS 3</div><div class="t m1 x4 h5 y22 ff5 fs2 fc1 sc0 ls0 ws0">5.4.4 权衡偏值和方差以最小化均方误差 . . . . . . . . . . . . . . 96</div><div class="t m1 x4 h5 y23 ff5 fs2 fc1 sc0 ls0 ws0">5.4.5 一致性 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97</div><div class="t m1 x3 h5 y24 ff5 fs2 fc1 sc0 ls0 ws0">5.5 最大似然估计 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97</div><div class="t m1 x4 h5 y25 ff5 fs2 fc1 sc0 ls0 ws0">5.5.1 条件对数似然和均方误差 . . . . . . . . . . . . . . . . . . . 99</div><div class="t m1 x4 h5 y26 ff5 fs2 fc1 sc0 ls0 ws0">5.5.2 最大似然的性质 . . . . . . . . . . . . . . . . . . . . . . . . 100</div><div class="t m1 x3 h5 y27 ff5 fs2 fc1 sc0 ls0 ws0">5.6 贝叶斯统计 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101</div><div class="t m1 x4 h5 y28 ff5 fs2 fc1 sc0 ls0 ws0">5.6.1 最大后验估计 . . . . . . . . . . . . . . . . . . . . . . . . . 103</div><div class="t m1 x3 h5 y29 ff5 fs2 fc1 sc0 ls0 ws0">5.7 监督学习算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104</div><div class="t m1 x4 h5 y2a ff5 fs2 fc1 sc0 ls0 ws0">5.7.1 概率监督学习 . . . . . . . . . . . . . . . . . . . . . . . . . 104</div><div class="t m1 x4 h5 y2b ff5 fs2 fc1 sc0 ls0 ws0">5.7.2 支持向量机 . . . . . . . . . . . . . . . . . . . . . . . . . . . 105</div><div class="t m1 x4 h5 y2c ff5 fs2 fc1 sc0 ls0 ws0">5.7.3 其他简单的监督学习算法 . . . . . . . . . . . . . . . . . . . 106</div><div class="t m1 x3 h5 y2d ff5 fs2 fc1 sc0 ls0 ws0">5.8 无监督学习算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108</div><div class="t m1 x4 h5 y2e ff5 fs2 fc1 sc0 ls0 ws0">5.8.1 PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109</div><div class="t m1 x4 h5 y2f ff5 fs2 fc1 sc0 ls0 ws0">5.8.2 k-均值聚类 . . . . . . . . . . . . . . . . . . . . . . . . . . . 111</div><div class="t m1 x3 h5 y30 ff5 fs2 fc1 sc0 ls0 ws0">5.9 随机梯度下降 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112</div><div class="t m1 x3 h5 y31 ff5 fs2 fc1 sc0 ls0 ws0">5.10 构建机器学习算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . 114</div><div class="t m1 x3 h5 y48 ff5 fs2 fc1 sc0 ls0 ws0">5.11 推动深度学习的挑战 . . . . . . . . . . . . . . . . . . . . . . . . . 115</div><div class="t m1 x4 h5 y49 ff5 fs2 fc1 sc0 ls0 ws0">5.11.1 维数灾难 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115</div><div class="t m1 x4 h5 y4a ff5 fs2 fc1 sc0 ls0 ws0">5.11.2 局部不变性和平滑正则化 . . . . . . . . . . . . . . . . . . . 115</div><div class="t m1 x4 h5 y4b ff5 fs2 fc1 sc0 ls0 ws0">5.11.3 流形学习 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118</div><div class="t m1 x2 h4 y36 ff3 fs2 fc1 sc0 ls0 ws0">6 深度前馈网络 121</div><div class="t m1 x3 h5 y37 ff5 fs2 fc1 sc0 ls0 ws0">6.1 例子：学习 XOR . . . . . . . . . . . . . . . . . . . . . . . . . . . 123</div><div class="t m1 x3 h5 y38 ff5 fs2 fc1 sc0 ls0 ws0">6.2 基于梯度的学习 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127</div><div class="t m1 x4 h5 y4c ff5 fs2 fc1 sc0 ls0 ws0">6.2.1 代价函数 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128</div><div class="t m1 x4 h5 y4d ff5 fs2 fc1 sc0 ls0 ws0">6.2.2 输出单元 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130</div><div class="t m1 x3 h5 y4e ff5 fs2 fc1 sc0 ls0 ws0">6.3 隐含单元 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138</div><div class="t m1 x4 h5 y4f ff5 fs2 fc1 sc0 ls0 ws0">6.3.1 修正线性单元极其扩展 . . . . . . . . . . . . . . . . . . . . 139</div><div class="t m1 x4 h5 y50 ff5 fs2 fc1 sc0 ls0 ws0">6.3.2 Logistic Sigmoid 与双曲正切函数 . . . . . . . . . . . . . . 141</div><div class="t m1 x4 h5 y51 ff5 fs2 fc1 sc0 ls0 ws0">6.3.3 其他隐含单元 . . . . . . . . . . . . . . . . . . . . . . . . . 141</div><div class="t m1 x3 h5 y52 ff5 fs2 fc1 sc0 ls0 ws0">6.4 结构设计 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143</div><div class="t m1 x4 h5 y53 ff5 fs2 fc1 sc0 ls0 ws0">6.4.1 通用近似性质和深度 . . . . . . . . . . . . . . . . . . . . . 143</div><div class="t m1 x4 h5 y54 ff5 fs2 fc1 sc0 ls0 ws0">6.4.2 其他结构上的考虑 . . . . . . . . . . . . . . . . . . . . . . . 145</div><div class="t m1 x3 h5 y55 ff5 fs2 fc1 sc0 ls0 ws0">6.5 反向传播和其他的微分算法 . . . . . . . . . . . . . . . . . . . . . . 147</div><div class="t m1 x4 h5 y56 ff5 fs2 fc1 sc0 ls0 ws0">6.5.1 计算图 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147</div><div class="t m1 x4 h5 y57 ff5 fs2 fc1 sc0 ls0 ws0">6.5.2 微积分中的链式法则 . . . . . . . . . . . . . . . . . . . . . 149</div><div class="t m1 x4 h5 y58 ff5 fs2 fc1 sc0 ls0 ws0">6.5.3 递归地使用链式法则来实现 BP . . . . . . . . . . . . . . . 150</div><div class="t m1 x4 h5 y59 ff5 fs2 fc1 sc0 ls0 ws0">6.5.4 全连接 MLP 中 BP 的计算 . . . . . . . . . . . . . . . . . . 151</div><div class="t m1 x4 h5 y5a ff5 fs2 fc1 sc0 ls0 ws0">6.5.5 符号到符号的导数 . . . . . . . . . . . . . . . . . . . . . . . 152</div><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a></div><div class="pi" data-data='{"ctm":[1.904762,0.000000,0.000000,1.904762,0.000000,0.000000]}'></div></div>

<div id="pf4" class="pf w0 h0" data-page-no="4"><div class="pc pc4 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/624704d962b5053d3c7dc90f/bg4.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">DRAFT</div><div class="t m1 x2 h5 y21 ff7 fs2 fc1 sc0 ls0 ws0">CONTENTS 4</div><div class="t m1 x4 h5 y22 ff5 fs2 fc1 sc0 ls0 ws0">6.5.6 一般化的 BP . . . . . . . . . . . . . . . . . . . . . . . . . . 153</div><div class="t m1 x4 h5 y23 ff5 fs2 fc1 sc0 ls0 ws0">6.5.7 实例：用于 MLP 训练的 BP . . . . . . . . . . . . . . . . . 155</div><div class="t m1 x4 h5 y24 ff5 fs2 fc1 sc0 ls0 ws0">6.5.8 复杂化 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156</div><div class="t m1 x4 h5 y25 ff5 fs2 fc1 sc0 ls0 ws0">6.5.9 深度学习社区以外的微分 . . . . . . . . . . . . . . . . . . . 157</div><div class="t m1 x4 h5 y26 ff5 fs2 fc1 sc0 ls0 ws0">6.5.10 高阶微分 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158</div><div class="t m1 x3 h5 y27 ff5 fs2 fc1 sc0 ls0 ws0">6.6 历史小记 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159</div><div class="t m1 x2 h4 y5b ff3 fs2 fc1 sc0 ls0 ws0">7 深度学习的正则化 162</div><div class="t m1 x3 h5 y5c ff5 fs2 fc1 sc0 ls0 ws0">7.1 参数范数惩罚 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163</div><div class="t m1 x4 h5 y5d ff5 fs2 fc1 sc0 ls0 ws0">7.1.1 L</div><div class="t m1 x7 h8 y5e ff9 fs4 fc1 sc0 ls0 ws0">2</div><div class="t m1 x8 h5 y5d ff4 fs2 fc1 sc0 ls0 ws0">参数正则化 . . . . . . . . . . . . . . . . . . . . . . . . . 164</div><div class="t m1 x4 h5 y5f ff5 fs2 fc1 sc0 ls0 ws0">7.1.2 L</div><div class="t m1 x7 h8 y60 ff9 fs4 fc1 sc0 ls0 ws0">1</div><div class="t m1 x8 h5 y5f ff4 fs2 fc1 sc0 ls0 ws0">参数正则化 . . . . . . . . . . . . . . . . . . . . . . . . . 166</div><div class="t m1 x3 h5 y61 ff5 fs2 fc1 sc0 ls0 ws0">7.2 作为约束的范数惩罚 . . . . . . . . . . . . . . . . . . . . . . . . . 168</div><div class="t m1 x3 h5 y62 ff5 fs2 fc1 sc0 ls0 ws0">7.3 正则化和欠约束问题 . . . . . . . . . . . . . . . . . . . . . . . . . 170</div><div class="t m1 x3 h5 y63 ff5 fs2 fc1 sc0 ls0 ws0">7.4 数据集增强 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171</div><div class="t m1 x3 h5 y64 ff5 fs2 fc1 sc0 ls0 ws0">7.5 噪声鲁棒性 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172</div><div class="t m1 x4 h5 y65 ff5 fs2 fc1 sc0 ls0 ws0">7.5.1 向输出目标注入噪声 . . . . . . . . . . . . . . . . . . . . . 173</div><div class="t m1 x3 h5 y66 ff5 fs2 fc1 sc0 ls0 ws0">7.6 半监督学习 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173</div><div class="t m1 x3 h5 y32 ff5 fs2 fc1 sc0 ls0 ws0">7.7 多任务学习 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174</div><div class="t m1 x3 h5 y33 ff5 fs2 fc1 sc0 ls0 ws0">7.8 提前终止 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175</div><div class="t m1 x3 h5 y34 ff5 fs2 fc1 sc0 ls0 ws0">7.9 参数绑定和参数共享 . . . . . . . . . . . . . . . . . . . . . . . . . 179</div><div class="t m1 x4 h5 y35 ff5 fs2 fc1 sc0 ls0 ws0">7.9.1 卷积神经网络 . . . . . . . . . . . . . . . . . . . . . . . . . 179</div><div class="t m1 x3 h5 y36 ff5 fs2 fc1 sc0 ls0 ws0">7.10 稀疏表示 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180</div><div class="t m1 x3 h5 y37 ff5 fs2 fc1 sc0 ls0 ws0">7.11 Bagging 和其他集成的方法 . . . . . . . . . . . . . . . . . . . . . . 181</div><div class="t m1 x3 h5 y38 ff5 fs2 fc1 sc0 ls0 ws0">7.12 dropout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183</div><div class="t m1 x3 h5 y4c ff5 fs2 fc1 sc0 ls0 ws0">7.13 对抗训练 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190</div><div class="t m1 x3 h5 y4d ff5 fs2 fc1 sc0 ls0 ws0">7.14 切面距离、正切传播和流形切分类 . . . . . . . . . . . . . . . . . . 191</div><div class="t m1 x2 h4 y3b ff3 fs2 fc1 sc0 ls0 ws0">8 深度模型中的优化 194</div><div class="t m1 x3 h5 y3c ff5 fs2 fc1 sc0 ls0 ws0">8.1 学习和优化有什么不同 . . . . . . . . . . . . . . . . . . . . . . . . 194</div><div class="t m1 x4 h5 y3d ff5 fs2 fc1 sc0 ls0 ws0">8.1.1 经验风险最小化 . . . . . . . . . . . . . . . . . . . . . . . . 195</div><div class="t m1 x4 h5 y3e ff5 fs2 fc1 sc0 ls0 ws0">8.1.2 替代损失函数和提前终止 . . . . . . . . . . . . . . . . . . . 196</div><div class="t m1 x4 h5 y3f ff5 fs2 fc1 sc0 ls0 ws0">8.1.3 （小）批算法 . . . . . . . . . . . . . . . . . . . . . . . . . 196</div><div class="t m1 x3 h5 y40 ff5 fs2 fc1 sc0 ls0 ws0">8.2 神经网络的优化挑战 . . . . . . . . . . . . . . . . . . . . . . . . . 200</div><div class="t m1 x4 h5 y41 ff5 fs2 fc1 sc0 ls0 ws0">8.2.1 病态 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200</div><div class="t m1 x4 h5 y42 ff5 fs2 fc1 sc0 ls0 ws0">8.2.2 局部极小值 . . . . . . . . . . . . . . . . . . . . . . . . . . . 200</div><div class="t m1 x4 h5 y43 ff5 fs2 fc1 sc0 ls0 ws0">8.2.3 高原，鞍点和其他平坦区域 . . . . . . . . . . . . . . . . . . 201</div><div class="t m1 x4 h5 y44 ff5 fs2 fc1 sc0 ls0 ws0">8.2.4 悬崖和梯度爆炸 . . . . . . . . . . . . . . . . . . . . . . . . 203</div><div class="t m1 x4 h5 y45 ff5 fs2 fc1 sc0 ls0 ws0">8.2.5 长期相关性 . . . . . . . . . . . . . . . . . . . . . . . . . . . 203</div><div class="t m1 x4 h5 y46 ff5 fs2 fc1 sc0 ls0 ws0">8.2.6 非精确梯度 . . . . . . . . . . . . . . . . . . . . . . . . . . . 204</div><div class="t m1 x4 h5 y47 ff5 fs2 fc1 sc0 ls0 ws0">8.2.7 局部和全局结构间的弱对应 . . . . . . . . . . . . . . . . . . 204</div><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a></div><div class="pi" data-data='{"ctm":[1.904762,0.000000,0.000000,1.904762,0.000000,0.000000]}'></div></div>

<div id="pf5" class="pf w0 h0" data-page-no="5"><div class="pc pc5 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/624704d962b5053d3c7dc90f/bg5.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">DRAFT</div><div class="t m1 x9 h4 y67 ff4 fs2 fc2 sc0 ls0 ws0">仅供学习使用，不得用于商业目的。https://github.com/exacity/deeplearningbook-chinese</div><div class="t m1 x2 h5 y21 ff7 fs2 fc1 sc0 ls0 ws0">CONTENTS 5</div><div class="t m1 x4 h5 y22 ff5 fs2 fc1 sc0 ls0 ws0">8.2.8 优化的理论限制 . . . . . . . . . . . . . . . . . . . . . . . . 205</div><div class="t m1 x3 h5 y23 ff5 fs2 fc1 sc0 ls0 ws0">8.3 基本算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206</div><div class="t m1 x4 h5 y24 ff5 fs2 fc1 sc0 ls0 ws0">8.3.1 随机梯度下降 . . . . . . . . . . . . . . . . . . . . . . . . . 206</div><div class="t m1 x4 h5 y25 ff5 fs2 fc1 sc0 ls0 ws0">8.3.2 动量 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207</div><div class="t m1 x4 h5 y26 ff5 fs2 fc1 sc0 ls0 ws0">8.3.3 Nesterov 动量 . . . . . . . . . . . . . . . . . . . . . . . . . 209</div><div class="t m1 x3 h5 y27 ff5 fs2 fc1 sc0 ls0 ws0">8.4 参数初始化策略 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210</div><div class="t m1 x3 h5 y28 ff5 fs2 fc1 sc0 ls0 ws0">8.5 自适应学习率的算法 . . . . . . . . . . . . . . . . . . . . . . . . . 213</div><div class="t m1 x4 h5 y29 ff5 fs2 fc1 sc0 ls0 ws0">8.5.1 AdaGrad . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214</div><div class="t m1 x4 h5 y2a ff5 fs2 fc1 sc0 ls0 ws0">8.5.2 RMSProp . . . . . . . . . . . . . . . . . . . . . . . . . . . 214</div><div class="t m1 x4 h5 y2b ff5 fs2 fc1 sc0 ls0 ws0">8.5.3 Adam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214</div><div class="t m1 x4 h5 y2c ff5 fs2 fc1 sc0 ls0 ws0">8.5.4 选择正确的优化算法 . . . . . . . . . . . . . . . . . . . . . 215</div><div class="t m1 x3 h5 y2d ff5 fs2 fc1 sc0 ls0 ws0">8.6 二阶近似方法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215</div><div class="t m1 x4 h5 y2e ff5 fs2 fc1 sc0 ls0 ws0">8.6.1 牛顿方法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215</div><div class="t m1 x4 h5 y2f ff5 fs2 fc1 sc0 ls0 ws0">8.6.2 共轭梯度 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216</div><div class="t m1 x4 h5 y30 ff5 fs2 fc1 sc0 ls0 ws0">8.6.3 BFGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218</div><div class="t m1 x3 h5 y31 ff5 fs2 fc1 sc0 ls0 ws0">8.7 优化技巧和元算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . 219</div><div class="t m1 x4 h5 y48 ff5 fs2 fc1 sc0 ls0 ws0">8.7.1 batch normalization . . . . . . . . . . . . . . . . . . . . . . 219</div><div class="t m1 x4 h5 y49 ff5 fs2 fc1 sc0 ls0 ws0">8.7.2 坐标下降 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221</div><div class="t m1 x4 h5 y4a ff5 fs2 fc1 sc0 ls0 ws0">8.7.3 Polyak 平均 . . . . . . . . . . . . . . . . . . . . . . . . . . 222</div><div class="t m1 x4 h5 y4b ff5 fs2 fc1 sc0 ls0 ws0">8.7.4 监督提前训练 . . . . . . . . . . . . . . . . . . . . . . . . . 222</div><div class="t m1 x4 h5 y68 ff5 fs2 fc1 sc0 ls0 ws0">8.7.5 设计有助于优化的模型 . . . . . . . . . . . . . . . . . . . . 224</div><div class="t m1 x4 h5 y69 ff5 fs2 fc1 sc0 ls0 ws0">8.7.6 连续方法和课程学习 . . . . . . . . . . . . . . . . . . . . . 224</div><div class="t m1 x2 h4 y38 ff3 fs2 fc1 sc0 ls0 ws0">9 卷积神经网络 227</div><div class="t m1 x3 h5 y4c ff5 fs2 fc1 sc0 ls0 ws0">9.1 卷积运算 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228</div><div class="t m1 x3 h5 y4d ff5 fs2 fc1 sc0 ls0 ws0">9.2 动机 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230</div><div class="t m1 x3 h5 y4e ff5 fs2 fc1 sc0 ls0 ws0">9.3 池化 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234</div><div class="t m1 x3 h5 y4f ff5 fs2 fc1 sc0 ls0 ws0">9.4 卷积与池化作为一种无限强的先验 . . . . . . . . . . . . . . . . . . 238</div><div class="t m1 x3 h5 y50 ff5 fs2 fc1 sc0 ls0 ws0">9.5 基本卷积函数的变体 . . . . . . . . . . . . . . . . . . . . . . . . . 239</div><div class="t m1 x3 h5 y51 ff5 fs2 fc1 sc0 ls0 ws0">9.6 结构化输出 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247</div><div class="t m1 x3 h5 y52 ff5 fs2 fc1 sc0 ls0 ws0">9.7 数据类型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248</div><div class="t m1 x3 h5 y53 ff5 fs2 fc1 sc0 ls0 ws0">9.8 高效的卷积算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248</div><div class="t m1 x3 h5 y54 ff5 fs2 fc1 sc0 ls0 ws0">9.9 随机或无监督的特征 . . . . . . . . . . . . . . . . . . . . . . . . . 249</div><div class="t m1 x3 h5 y55 ff5 fs2 fc1 sc0 ls0 ws0">9.10 卷积神经网络的神经科学基础 . . . . . . . . . . . . . . . . . . . . 250</div><div class="t m1 x3 h5 y56 ff5 fs2 fc1 sc0 ls0 ws0">9.11 卷积神经网路与深度学习的历史 . . . . . . . . . . . . . . . . . . . 256</div><div class="t m1 x2 h4 y44 ff3 fs2 fc1 sc0 ls0 ws0">10 序列建模：循环和递归网络 257</div><div class="t m1 x3 h5 y45 ff5 fs2 fc1 sc0 ls0 ws0">10.1 展开计算图 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258</div><div class="t m1 x3 h5 y46 ff5 fs2 fc1 sc0 ls0 ws0">10.2 循环神经网络 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260</div><div class="t m1 x4 h5 y47 ff5 fs2 fc1 sc0 ls0 ws0">10.2.1 Teacher Forcing 和输出循环网络 . . . . . . . . . . . . . . . 263</div><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a><a class="l" rel='nofollow' onclick='return false;'><div class="d m2"></div></a></div><div class="pi" data-data='{"ctm":[1.904762,0.000000,0.000000,1.904762,0.000000,0.000000]}'></div></div>