本站消息

站长简介


前每日优鲜python全栈开发工程师,自媒体达人,逗比程序猿,钱少话少特宅,我的公众号:想吃麻辣香锅

  python大神匠心打造,零基础python开发工程师视频教程全套,基础+进阶+项目实战,包含课件和源码

  出租广告位,需要合作请联系站长



+关注
已关注

分类  

暂无分类

标签  

暂无标签

日期归档  

2020-06(24)

2020-07(28)

2020-08(34)

2020-09(64)

2020-10(64)

MiniFlow -- 2.什么是神经网络

发布于2021-05-29 13:35     阅读(1061)     评论(0)     点赞(17)     收藏(5)


0

1

2

3

4

5

6



在这里插入图片描述

1.神经网络

如上图,这是一个简单的神经网络,神经网络就是一个包含数学运算的流程图(Graph),这些数学运算包括矩阵运算,还有激活函数。这个图表包括节点/神经元(neurons/nodes),链接/边界线(links/edges)。
每一层的节点的就是使用上一层的的输出作为本层的输入进行数学运算,当然不包括输入层次(input layer),因为输入层不进行运算
比如,每一个节点进行f(x,y)的运算,x和y都是将来自上一层的输出作为本层的输入。因此,每一层都会生成一个输出值,输出层除外(output layer)
在输入层和输出层之间的所有层都叫做隐藏层(Hidden layer)

2.前向传播

从输入层开始,将一些数值经过每一层的数学运算,这个网络最后输出一个值,这个过程就叫做前向传播(Forward pass),如下这个简单的流程,最后输出30
在这里插入图片描述

3. Graphs

神经元(neurons/nodes)和链接(edges/links)构成了一个图结构(graph structure)。
定义一个Graphs一般需要两个步骤:

  1. 定义图的节点和链接
  2. 在Graphs里面传递数值

4.Neurons and MiniFlow

我们使用python来构建我们的MiniFlow

class Neuron:
    def __init__(self):
        # Properties will go here!

正如我们所知,每一个节点都从多个其他的节点接受输入数据作为输入,我们也知道,每一个节点都生成一个输出,然后传给其他节点,那么,让我们添加两个list,一个用来存储输入节点,一个用来存储输出节点

class Neuron:
    def __init__(self, inbound_neurons=[]):
        # Neuron from which this Neuron receives values
        self.inbound_neurons = inbound_neurons
        # Neuron to which this Neuron passes values
        self.outbound_neurons = []
        # Add this node as an outbound node on its inputs.
        for n in self.inbound_neurons:
            n.outbound_neurons.append(self)

每一个节点都要将输入的值根据本节点的数学关系,计算出一个值作为他的输出。现在我们初始化 value为None

class Neuron:
    def __init__(self, inbound_neurons=[]):
        # Neuron from which this Neuron receives values
        self.inbound_neurons = inbound_neurons
        # Neuron to which this Neuron passes values
        self.outbound_neurons = []
        # A calculated value
        self.value = None

每一个节点都需要能够向前计算输出,向后更新权重,因此,这里我们添加两个占位符,一个是向前计算,一个是后向传播

raise NotImplemented 表示子类必须实现该方法

class Neuron:
    def __init__(self, inbound_neurons=[]):
        # Neuron from which this Neuron receives values
        self.inbound_neurons = inbound_neurons
        # Neuron to which this Neuron passes values
        self.outbound_neurons = []
        # A calculated value
        self.value = None
        # Add this node as an outbound node on its inputs.
        for n in self.inbound_neurons:
            n.outbound_neurons.append(self)

    def forward(self):
        """
        Forward propagation.

        Compute the output value based on `inbound_neurons` and
        store the result in self.value.
        """
        raise NotImplemented

    def backward(self):
        """
        Backward propagation.

        You'll compute this later.
        """
        raise NotImplemented

5. 神经元的计算

因为不同的节点执行不同的前向运算和反向传播,上面只是定义了一个父类,对于不同的节点我们还需要定义不同的子类来细化特定的运算,比如这里我们定义输入层

class Input(Neuron):
    def __init__(self):
        # An Input neuron has no inbound neurons,
        # so no need to pass anything to the Neuron instantiator.
        Neuron.__init__(self)

    # NOTE: Input neuron is the only node where the value
    # may be passed as an argument to forward().
    #
    # All other neuron implementations should get the value
    # of the previous neuron from self.inbound_neurons
    #
    # Example:
    # val0 = self.inbound_neurons[0].value
    def forward(self, value=None):
        # Overwrite the value if one is passed in.
        if value:
            self.value = value

不同于其他Neuron的子类,Input子类不需要计算任何事情,Input子类只是保持一些值,比如数据特征/特征

Add 子类

class Add(Neuron):
    def __init__(self, x, y):
        Neuron.__init__(self, [x, y])

    def forward(self):
        """
        You'll be writing math here in the first quiz!
        """

6.定义Graphs和计算流程

定义神经网络最重要的之一就是定义计算图,就是神经网络的节点之间的计算关系,如下图,每个点就代表一个neuron,他们之间的链接线表示计算关系
在这里插入图片描述
为了定义我们自己的网络,我们得定义神经元之间的操作关系,也就是计算关系,根据上级节点的输出作为本节点的输入,我们需要将这些节点展平,为拓扑逻辑关系,如下

x, y = Input(), Input()

sorted_neurons = topological_sort(feed_dict={x: 10, y: 20})

我们之后会提供topological_sort的具体代码

我们也会提供forward_pass方法,用来进行前向运算

def forward_pass(output_neuron, sorted_neurons):
    """
    Performs a forward pass through a list of sorted neurons.

    Arguments:

        `output_neuron`: The output neuron of the graph (no outgoing edges).
        `sorted_neurons`: a topologically sorted list of neurons.

    Returns the output neuron's value
    """

    for n in sorted_neurons:
        n.forward()

    return output_neuron.value

我们接下来给出 nn.py 和 miniflow.py.代码,下面的代码只是先展示一下MiniFlow的工作流程

nn.py

"""
This script builds and runs a graph with miniflow.

There is no need to change anything to solve this quiz!

However, feel free to play with the network! Can you also
build a network that solves the equation below?

(x + y) + y
"""

from miniflow import *

x, y = Input(), Input()

f = Add(x, y)

feed_dict = {x: 10, y: 5}

sorted_neurons = topological_sort(feed_dict)
output = forward_pass(f, sorted_neurons)

print("{} + {} = {} (according to miniflow)".format(feed_dict[x], feed_dict[y], output))

miniflow.py

"""
You need to change the Add() class below.
"""

class Neuron:
    def __init__(self, inbound_neurons=[]):
        # Neurons from which this Node receives values
        self.inbound_neurons = inbound_neurons
        # Neurons to which this Node passes values
        self.outbound_neurons = []
        # A calculated value
        self.value = None
        # Add this node as an outbound node on its inputs.
        for n in self.inbound_neurons:
            n.outbound_neurons.append(self)

    # These will be implemented in a subclass.
    def forward(self):
        """
        Forward propagation.

        Compute the output value based on `inbound_neurons` and
        store the result in self.value.
        """
        raise NotImplemented

    def backward(self):
        """
        Backward propagation.

        Compute the gradient of the current node with respect
        to the input neurons. The gradient of the loss with respect
        to the current neuron should already be computed in the `gradients`
        attribute of the output neurons.
        """
        raise NotImplemented


class Input(Neuron):
    def __init__(self):
        # an Input neuron has no inbound nodes,
        # so no need to pass anything to the Node instantiator
        Neuron.__init__(self)

    # NOTE: Input node is the only node where the value
    # is passed as an argument to forward().
    #
    # All other neuron implementations should get the value
    # of the previous neurons from self.inbound_neurons
    #
    # Example:
    # val0 = self.inbound_neurons[0].value
    def forward(self, value=None):
        # Overwrite the value if one is passed in.
        if value:
            self.value = value


class Add(Neuron):
    def __init__(self, x, y):
        Neuron.__init__(self, [x, y])

    def forward(self):
        """
        Set the value of this neuron to the sum of it's inbound_nodes.
        
        Your code here!
        """


"""
No need to change anything below here!
"""


def topological_sort(feed_dict):
    """
    Sort generic nodes in topological order using Kahn's Algorithm.

    `feed_dict`: A dictionary where the key is a `Input` node and the value is the respective value feed to that node.

    Returns a list of sorted nodes.
    """

    input_neurons = [n for n in feed_dict.keys()]

    G = {}
    neurons = [n for n in input_neurons]
    while len(neurons) > 0:
        n = neurons.pop(0)
        if n not in G:
            G[n] = {'in': set(), 'out': set()}
        for m in n.outbound_neurons:
            if m not in G:
                G[m] = {'in': set(), 'out': set()}
            G[n]['out'].add(m)
            G[m]['in'].add(n)
            neurons.append(m)

    L = []
    S = set(input_neurons)
    while len(S) > 0:
        n = S.pop()

        if isinstance(n, Input):
            n.value = feed_dict[n]

        L.append(n)
        for m in n.outbound_neurons:
            G[n]['out'].remove(m)
            G[m]['in'].remove(n)
            # if no other incoming edges add to S
            if len(G[m]['in']) == 0:
                S.add(m)
    return L


def forward_pass(output_neuron, sorted_neurons):
    """
    Performs a forward pass through a list of sorted neurons.

    Arguments:

        `output_neuron`: A neuron in the graph, should be the output neuron (have no outgoing edges).
        `sorted_neurons`: a topologically sorted list of neurons.

    Returns the output neuron's value
    """

    for n in sorted_neurons:
        n.forward()

    return output_neuron.value

原文链接:https://blog.csdn.net/xf8964/article/details/117308378




0

1

2

3

4

5

6

7

8

9



所属网站分类: 技术文章 > 博客

作者:djjdf

链接:https://www.pythonheidong.com/blog/article/977677/f012d82423f3cd60845e/

来源:python黑洞网

任何形式的转载都请注明出处,如有侵权 一经发现 必将追究其法律责任

17 0
收藏该文
已收藏

评论内容:(最多支持255个字符)