机器学习-人为设置函数方法和神经网络方法解决智能五子棋问题

2  智能决策

2.1 博弈树模型算法

2.1.1 全局估算函数

此次项目中评估函数有两种：

    （1）人为设定函数方法：更具人的经验，对一些特定的棋形在棋盘上进行检索。并且计数，最后赋予相应权值求和得到对棋盘的评价值。典型的棋形有“活一”“活二”“活三”“冲四”“成龙”。越接近于五子连珠的棋形应有越大的权重。由于先手优势，同一种棋形黑棋的权重的绝对值应略大于白棋的权重。博弈双方的权重的符号应相反。不失一般性，黑棋的权重设置为正。

    （2）人工神经网络方法：用225个输入节点1个输出节点的神经网络对棋局进行评估。考虑到笔记本电脑计算条件本次实验中采用三层全连接网络结构，中间节点数为10。255个输入节点对应15*15棋盘上的所有点。如该点为黑棋输入为1，白棋输入-1，无子输入0。神经网络初始参数有高斯分布确定，再用遗传算法优化

2.1.2 极大极小算法

假设对弈的甲乙双方均为理性人，则对于甲走的每一步，乙都会选择下使得自己优势最大，甲的劣势最大的一步，甲下棋时亦然。例如在图6中在A棋局情况下，如果甲选择走B棋局， 那么乙将选择走对甲不利的E棋局；如果甲选择走 C棋局，乙将选择走G棋局；如果甲选择走D棋局， 那么乙将选择走K棋局……所以，综合来看，甲应该选择走B棋局，这样对自己最有利，这种算法就是极大极小算法。

         图8 博弈树模型

2.2  博弈树模型算法优化与实现

2.2.1 增量分析法

   在用人为设定函数计算棋局价值数值时，如果每次都遍历整个棋盘，则需要大量时间，事实上，只需计算出当前这一步落子在四个方向上的价值和未落子时该点四个方向上的价值之差再加上原来棋盘的价值及即可求得落子之后期盼的总价值

2.2.2 局部性原理

控制博弈树的层次和每层展开的节点数目，对反应速度和智能程度至关重要，根据人类棋手的经验，五子棋问题中，新下的棋与其余棋子的距离和该点的价值相关，距离越远，对整个棋盘影响越小，距离越近影响越大。这就是局部性原理，运用它可以选择有“潜力”的点进行展开，缩小搜索范围，提高搜索效率。

2.2.3 α-β剪枝

   很多时候遍历整个博弈树是没有必要的，如图6中，在A棋局下轮到甲走棋，甲首先搜索叶子节点E=7，F=11，计算得到B 棋局的倒推值为7；然后，当甲继续搜索到节点G= 1时，发现C(C≤1)必定小于B。也就是说，走第二 个子节点C棋局肯定不如走第一个子节点曰棋局 有利，那么子节点C棋局往下的其它节点棋局(H、I)就不用再搜索了，此过程就称为α-β剪枝。，运用α-β剪枝可以加快搜索速度，且党棋局越明郎，搜索速度越快。

2.3 实验结果与分析

2.3.1 人为设定函数的机器棋手表现

表1 机器五子棋的测试结果

2.3.1采用神经网络和遗传算法的机器棋手表现

运用遗传算法神经网络训练出的机器棋手表现极为不佳分析原因有以下两点：

（1）种群数量太小，基因多样性丢失：在种群数量20，突变率为0.1的实验中，实验进行到第28代后，每次棋局都是变得一模一样，主要原因是种群数量太小，次要原因是突变率太小

图9 种群20第28代出现多样性丢失

（2）训练代数太小，未学到智能性：在种群数量为100，突变率为0.3的实验中，经过长达两小时的训练到达第33代时，虽然没有出现基因多样性消失的现象，但机器棋手仍未表现出智能的迹象，主要原因是计算能力有限，短时间内训练代数太少，次要原因是网络初始参数不带有任何先天经验，从零开始学习比较费时

图10 种群100第33代仍未有智能表现

import time
import numpy as np
import copy
import pygame
import sys


#首先是‘活一’，‘冲四’等特定棋形的检测模板设置
flex1 = [[0, 1, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0], [0, 0, 0, 1, 0, 0, ], [0, 0, 0, 0, 1, 0]]
Flex1 = [[0, -1, 0, 0, 0, 0], [0, 0, -1, 0, 0, 0], [0, 0, 0, -1, 0, 0, ], [0, 0, 0, 0, -1, 0]]
flex2 = [[0, 1, 1, 0, 0, 0], [0, 0, 1, 1, 0, 0], [0, 0, 0, 1, 1, 0], [0, 1, 0, 1, 0, 0], [0, 0, 1, 0, 1, 0],
         [0, 1, 0, 0, 1, 0]]
Flex2 = [[0, -1, -1, 0, 0, 0], [0, 0, -1, -1, 0, 0], [0, 0, 0, -1, -1, 0], [0, -1, 0, -1, 0, 0], [0, 0, -1, 0, -1, 0]]
flex3 = [[0, 1, 1, 1, 0, 0], [0, 1, 1, 0, 1, 0], [0, 0, 1, 1, 1, 0], [0, 1, 0, 1, 1, 0]]
Flex3 = [[0, -1, -1, -1, 0, 0], [0, -1, -1, 0, -1, 0], [0, 0, -1, -1, -1, 0], [0, -1, 0, -1, -1, 0]]
flex4 = [[0, 1, 1, 1, 1, 0]]
Flex4 = [[0, -1, -1, -1, -1, 0]]
white_win = [[1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 0], [0, 1, 1, 1, 1, 1, ], [-1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, -1],
             [2, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 2]]
black_win = [[-1, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, 0], [0, -1, -1, -1, -1, -1, ], [1, -1, -1, -1, -1, -1],
             [-1, -1, -1, -1, -1, 1], [2, -1, -1, -1, -1, -1], [-1, -1, -1, -1, -1, 2]]

block2 = [[2, 1, 1, 0, 0, 0, ], [0, 0, 0, 1, 1, 2], [-1, 1, 1, 0, 0, 0, ], [0, 0, 0, 1, 1, -1]]
Block2 = [[2, -1, -1, 0, 0, 0, ], [0, 0, 0, -1, -1, 2], [1, -1, -1, 0, 0, 0, ], [0, 0, 0, -1, -1, 1]]
block3 = [[2, 1, 1, 1, 0, 0, ], [0, 0, 1, 1, 1, 2], [-1, 1, 1, 1, 0, 0, ], [0, 0, 1, 1, 1, -1], [2, 1, 1, 0, 1, 0, ],
          [0, 1, 0, 1, 1, 2], [-1, 1, 1, 0, 1, 0, ], [0, 1, 0, 1, 1, -1]]
Block3 = [[2, -1, -1, -1, 0, 0, ], [0, 0, -1, -1, -1, 2], [1, -1, -1, -1, 0, 0, ], [0, 0, -1, -1, -1, 1],
          [2, -1, -1, 0, -1, 0, ], [0, -1, 0, -1, -1, 2], [1, -1, -1, 0, -1, 0, ], [0, -1, 0, -1, -1, 1]]
block4 = [[2, 1, 1, 1, 1, 0, ], [0, 1, 1, 1, 1, 2], [-1, 1, 1, 1, 1, 0, ], [0, 1, 1, 1, 1, -1], [2, 1, 1, 0, 1, 1],
          [2, 1, 0, 1, 1, 1], [2, 1, 1, 1, 0, 1]]
Block4 = [[2, -1, -1, -1, -1, 0, ], [0, -1, -1, -1, -1, 2], [1, -1, -1, -1, -1, 0, ], [0, -1, -1, -1, -1, 1],
          [2, -1, -1, 0, -1, -1], [2, -1, 0, -1, -1, -1], [2, -1, -1, -1, 0, -1]]

#游戏界面的参数设置
screen_size = (671, 671)
WIDTH = 720
HEIGHT = 720
width = 44
GRID_WIDTH = WIDTH // 20
WHITE = (255, 250, 255)
BLACK = (0, 0, 0)
GREEN = (0, 0xff, 0)
RED = (0xff, 0, 0)
bgcolor = (255, 255, 255)
color = [BLACK, WHITE]
pygame.init()
icon = pygame.image.load("image/五子棋.jpeg")
pygame.display.set_icon(icon)
screen = pygame.display.set_mode(screen_size, 0, 32)
pygame.display.set_caption('欢乐五子棋')
clock = pygame.time.Clock()
bg = pygame.image.load("image/checker.png")

#函数作用同函数名
def draw_check(surf):
    screen.fill(bgcolor)
    surf.blit(bg, (0, 0))


def draw_checker(surf, x, y, color):
    if color == 1:  # 执白棋
        pygame.draw.circle(surf, BLACK, ((x) * width + 27, (y) * width + 26), 20, 0)
    if color == -1:  # 执黑棋
        pygame.draw.circle(surf, WHITE, ((x) * width + 27, (y) * width + 26), 20, 0)


def winner_check(chessboard, ones_turn, board_size, chesser):
    counter_x, counter_y, counter_1, counter_2 = 0, 0, 0, 0
    for i in range(board_size):
        if chessboard[ones_turn[0]][i] == chesser:
            counter_x += 1
        else:
            counter_x = 0
        if counter_x == 5:
            return 1

        if chessboard[i][ones_turn[1]] == chesser:
            counter_y += 1
        else:
            counter_y = 0
        if counter_y == 5:
            return 1

    tip = ones_turn.index(max(ones_turn))
    if tip == 0:
        i, j = ones_turn[0] - ones_turn[1], 0
    else:
        i, j = 0, ones_turn[1] - ones_turn[0]
    while j < board_size and i < board_size:
        if chessboard[i][j] == chesser:
            counter_1 += 1
        else:
            counter_1 = 0
        if counter_1 == 5:
            return 1
        i += 1
        j += 1

    if ones_turn[0] + ones_turn[1] >= board_size + 1:
        i, j = ones_turn[0] + ones_turn[1] - (board_size + 1), board_size + 1
    else:
        i, j = 0, ones_turn[0] + ones_turn[1]

    while j >= 0 and i <= board_size + 1:
        if chessboard[i][j] == chesser:
            counter_2 += 1
        else:
            counter_2 = 0
        if counter_2 == 5:
            return 1
        i += 1
        j -= 1
    return 0

#得分计算
def score_count(windows, black_score, white_score):
    weight = [1, 1, 20, 20, 400, 50000, 10000, 500000, ]
    ex = 0.3
    if windows in flex1:
        white_score += weight[0]
    if windows in Flex1:
        black_score -= ex * weight[0]
    if windows in block2:
        white_score += weight[1]
    if windows in Block2:
        black_score -= ex * weight[1]
    if windows in flex2:
        white_score += weight[2]
    if windows in Flex2:
        black_score -= ex * weight[2]
    if windows in block3:
        white_score += weight[3]
    if windows in Block3:
        black_score -= ex * weight[3]
    if windows in flex3:
        white_score += weight[4]
    if windows in Flex3:
        black_score -= ex * weight[4]
    if windows in block4:
        white_score += weight[5]
    if windows in Block4:
        black_score -= ex * weight[5]
    if windows in flex4:
        white_score += weight[6]
    if windows in Flex4:
        black_score -= ex * weight[6]

    if windows in white_win:
        white_score += weight[7]
    if windows in black_win:
        black_score -= ex * weight[7]
    return black_score, white_score

#棋盘得分计算
def eval_chessboard(chessboard, ones_turn, old_score):  # ones_turn,old_score
    white_score, black_score = 0, 0
    #   x y axis
    for i in range(chessboard.shape[0] - 5):
        windows = list(chessboard[i:i + 6, ones_turn[1]])
        black_score, white_score = score_count(windows, black_score, white_score)
        windows = list(chessboard[ones_turn[0], i: i + 6, ])
        black_score, white_score = score_count(windows, black_score, white_score)
    tip = ones_turn.index(max(ones_turn))
    if tip == 0:
        i, j = ones_turn[0] - ones_turn[1], 0
    else:
        i, j = 0, ones_turn[1] - ones_turn[0]
    while j <= board_size - 4 and i <= board_size - 4:
        windows.clear()
        for k in range(6):
            windows.append(chessboard[i + k][j + k])
        black_score, white_score = score_count(windows, black_score, white_score)
        j += 1
        i += 1

    if ones_turn[0] + ones_turn[1] >= board_size + 1:
        i, j = ones_turn[0] + ones_turn[1] - (board_size + 1), board_size + 1
    else:
        i, j = 0, ones_turn[0] + ones_turn[1]

    while j >= 5 and i <= board_size - 4:
        windows.clear()
        for k in range(6):
            windows.append(chessboard[i + k][j - k])
        black_score, white_score = score_count(windows, black_score, white_score)
        i += 1
        j -= 1

    return old_score + white_score + black_score

#得分增益
def compute_score(chessboard, ones_turn, old_score):
    t1 = eval_chessboard(chessboard, ones_turn, old_score)
    ori_chessboard = copy.deepcopy(chessboard)
    ori_chessboard[ones_turn[0]][ones_turn[1]] = 0
    t0 = eval_chessboard(ori_chessboard, ones_turn, old_score)
    return t1 - t0 + old_score


# 更新搜索边界，这是用来优化减小搜索空间的
def update_boundary(boundary, i, j, size):  # size max axis from 1

    width = 2
    if i == 15:
        width = width
    if boundary[0] == 0:
        boundary[0], boundary[1] = j, j
        boundary[2], boundary[3] = i, i
    if boundary[2] >= i - width:
        if i - width >= 1:
            boundary[2] = i - width
        else:
            boundary[2] = 1

    if boundary[3] <= i + width:
        if i + width <= size:
            boundary[3] = i + width
        else:
            boundary[3] = size

    if boundary[0] >= j - width:
        if j - width >= 1:
            boundary[0] = j - width
        else:
            boundary[0] = 1
    if boundary[1] <= j + width:
        if j + width <= size:
            boundary[1] = j + width
        else:
            boundary[1] = size

    return boundary

# 树搜索
def tree_search(chessboard, boundary, search_degree, type, pre_step, pre_score, apha, beta):
    # boundry=(l,r,t,d)
    # boundry should be a tuple with four feature represent the boundary square
    score_list, position_list, back_score_list = [], [], []

    if search_degree == 0:
        """score_1=compute_score(chessboard,pre_step,pre_score)
        chessboard[pre_step[0]][pre_step[1]]=0
        score_0=compute_score(chessboard,pre_step,pre_score)"""
        return compute_score(chessboard, pre_step, pre_score), pre_step  # maybe?

    for i in range(boundary[2], boundary[3] + 1):
        for j in range(boundary[0], boundary[1] + 1):
            if chessboard[i][j] == 0:
                if type == 1:
                    chessboard[i][j] = 1
                    tem_score = compute_score(chessboard, (i, j), pre_score)
                    score_list.append((tem_score, i, j))
                else:
                    chessboard[i][j] = -1
                    tem_score = compute_score(chessboard, (i, j), pre_score)
                    score_list.append((tem_score, i, j))
                chessboard[i][j] = 0
    score_list.sort(key=lambda x: x[0], reverse=(type == 1))
    for k in range(8):
        i, j, tem_score = score_list[k][1], score_list[k][2], score_list[k][0]
        new_boundary = copy.deepcopy(boundary)
        new_boundary = update_boundary(new_boundary, i, j, chessboard.shape[0] - 1)

        tem0, tem1 = tree_search(chessboard, new_boundary, search_degree - 1, -type, (i, j), tem_score, apha, beta)
        back_score_list.append(tem0)

        if tem0 > apha:
            apha = tem0
        if tem0 < beta:
            beta = tem0
        chessboard[i][j] = 0
        if beta <= apha:
            return tem0, (i, j)

        if type == 1:
            return max(back_score_list), score_list[back_score_list.index(max(back_score_list))][1], \
                   score_list[back_score_list.index(max(back_score_list))][2]
        if type == -1:
            return min(back_score_list), score_list[back_score_list.index(min(back_score_list))][1], \
                   score_list[back_score_list.index(min(back_score_list))][2]





score = 0
score_max = 0
board_size = 15  # needs boundary filled by 2
boundary = [0, 0, 0, 0]
# flex1,Flex1,block2,Block2,flex2,Flex2,block3,Block3,flex3,Flex3,block4,Block4,w1,W1
chance_count = np.zeros((1, board_size))
chessboard = np.zeros((board_size + 2, board_size + 2)) + 2
chessboard[1:-1, 1:-1] -= 2
# black_chess = eval(input()) #
black_chess = 1
out = 0
while 1:

    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            sys.exit()
        if event.type == pygame.MOUSEBUTTONDOWN:
            pos = event.pos
            human_turn1 = (int(round((pos[0] - 27) / width)), int(round((pos[1] - 26) / width)))
            human_turn = human_turn1[0] + 1, human_turn1[1] + 1
            if chessboard[human_turn[0]][human_turn[1]] == 0:
                chessboard[human_turn[0]][human_turn[1]] = 1
                boundary = update_boundary(boundary, human_turn[0], human_turn[1], board_size)
                if winner_check(chessboard, human_turn, board_size, 1) == 1:
                    pygame.display.set_caption('哇，妈妈赢了，腻害腻害')
                    out = 1
                    break
            for i in range(1, board_size + 1):
                for j in range(1, board_size + 1):
                    draw_checker(screen, i - 1, j - 1, chessboard[i][j])
            pygame.display.update()
            clock.tick(50)

            score = compute_score(chessboard, human_turn, score)
            score_max, robot_turn = tree_search(chessboard, boundary, 2, -1, (0, 0), score, -100000000, 100000000)
            chessboard[robot_turn[0]][robot_turn[1]] = -1
            score = compute_score(chessboard, robot_turn, score)
            boundary = update_boundary(boundary, robot_turn[0], robot_turn[1], board_size)
            if winner_check(chessboard, robot_turn, board_size, -1) == 1:
                print("robot win")
                pygame.display.set_caption('承让承让了妈妈')
                out = 1
                break

        draw_check(screen)
        for i in range(1, board_size + 1):
            for j in range(1, board_size + 1):
                draw_checker(screen, i - 1, j - 1, chessboard[i][j])
        pygame.display.update()
        if out == 1:
            time.sleep(3)
            sys.exit()
        clock.tick(50)

import torch
import random
import copy
import pygame
import time

chessboard = torch.randn((255,1))
ensor =  torch.Tensor(100,100,255,)
torch.nn.init.normal_(ensor)

screen_size = (671,671)
WIDTH = 720
HEIGHT = 720
width = 44
GRID_WIDTH = WIDTH // 20
WHITE = (255, 250, 255)
BLACK = (0, 0, 0)
GREEN = (0, 0xff, 0)
RED = (0xff, 0, 0)
bgcolor = (255,255,255)
color = [BLACK,WHITE]
pygame.init()
screen_size = (671, 671)
pygame.init()
icon = pygame.image.load("image/五子棋.jpeg")
pygame.display.set_icon(icon)
screen = pygame.display.set_mode(screen_size, 0, 32)
pygame.display.set_caption('欢乐五子棋')
clock = pygame.time.Clock()
bg = pygame.image.load("image/checker.png")
fname ="weight.txt"

class F_c(torch.nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(F_c, self).__init__()
        self.fc1 = torch.nn.Linear(input_size, hidden_size,)
        self.sigmoid = torch.nn.Sigmoid()
        self.fc2 = torch.nn.Linear(hidden_size, num_classes,)

    def forward(self, x):
        out = self.fc1(x)
        out = self.sigmoid(out)
        out = self.fc2(out)
        return out


def draw_check(surf):
    screen.fill(bgcolor)
    surf.blit(bg, (0, 0))


def draw_checker(surf,x,y,color):

    if color == 1:#执白棋
        pygame.draw.circle( surf, BLACK, ((x) * width+27, (y) * width+26), 20, 0)
    if color == -1:#执黑棋
        pygame.draw.circle( surf, WHITE, ((x) * width+27, (y) * width+26), 20, 0)

def winner_check(chessboard,ones_turn,board_size,chesser):
    counter_x,counter_y,counter_1,counter_2 =0,0,0,0
    for i in range(board_size):
            if chessboard[ones_turn[0]][i]==chesser:
                counter_x += 1
            else:
                counter_x= 0
            if counter_x==5:
                return 1

            if chessboard[i][ones_turn[1]]==chesser:
                counter_y += 1
            else:
                counter_y= 0
            if counter_y==5:
                return 1

    tip =ones_turn.index(max(ones_turn))
    if tip==0:
        i,j=ones_turn[0]-ones_turn[1],0
    else:
        i,j =0,ones_turn[1]-ones_turn[0]
    while j<board_size and i<board_size:
        if chessboard[i][j]==chesser:
            counter_1+=1
        else:
            counter_1=0
        if counter_1==5:
            return 1
        i+=1
        j+=1

    if ones_turn[0]+ones_turn[1]>=board_size:
        i,j=ones_turn[0]+ones_turn[1]-(board_size-1),board_size-1
    else:
        i,j=0,ones_turn[0]+ones_turn[1]


    while j>=0 and  i<=board_size-1:
        if chessboard[i][j]==chesser:
            counter_2+=1
        else:
            counter_2=0
        if counter_2==5:
            return 1
        i+=1
        j-=1
    return 0
def update_boundary(boundary,i,j,size):# size max axis from 1

    width =2
    if boundary[0] == -1:
        boundary[0], boundary[1] = j, j
        boundary[2], boundary[3] = i, i
    if boundary[0]>=j-width:
        if j-width>=0:
            boundary[0]=j-width
        else:
            boundary[1]=0
    if boundary[1]<=j+width:
        if j+width<size-1:
            boundary[1]=j+width
        else:
            boundary[1]=size-1

    if boundary[2]>=i-width:
        if i-width>=0:
            boundary[2]=i-width
        else:
            boundary[2]=0

    if boundary[3]<=i+width:
        if i+width<size-1:
            boundary[3]=i+width
        else:
            boundary[3]=size-1

    return boundary

def compute_score(chessboard,weight_tensor):#the latter should be the object lof F_c
    chessboard_tensor=chessboard.clone().detach()
    #chessboard_tensor=torch.cuda.FloatTensor(chessboard_tensor)
    chessboard_tensor = chessboard_tensor.to(device)
    chessboard=chessboard-2
    chessboard_tensor.resize_(225)
    #ues torch.nn
    x =weight_tensor.forward(chessboard_tensor)
    return x



def tree_search(chessboard,weight_tensor,boundary,pre_step,search_degree,type,apha,beta):
    # boundry=(l,r,t,d)
    # boundry should be a tuple with four feature represent the boundary square
    score_list, position_list, back_score_list = [], [], []
    if search_degree == 0:
        return compute_score(chessboard,weight_tensor), pre_step  # maybe?

    for i in range(boundary[2], boundary[3]+1 ):
        for j in range(boundary[0], boundary[1]+1 ):
            if chessboard[i][j] == 2:
                if type == 1:
                    chessboard[i][j] = 1
                    tem_score = compute_score(chessboard,weight_tensor)
                    score_list.append((tem_score, i, j))
                else:
                    chessboard[i][j] = -1
                    tem_score = compute_score(chessboard,weight_tensor)
                    score_list.append((tem_score, i, j))
                chessboard[i][j] =2
    score_list.sort(key=lambda x: x[0], reverse=(type == 1))
    for k in range(min(8,len(score_list))):
        i, j, tem_score = score_list[k][1], score_list[k][2], score_list[k][0]
        new_boundary = copy.deepcopy(boundary)
        new_boundary = update_boundary(new_boundary, i, j, chessboard.shape[0] - 1)
        tem0, tem1 = tree_search(chessboard, weight_tensor,new_boundary,(i,j), search_degree - 1, -type, apha, beta)
        back_score_list.append(tem0)

        if tem0 > apha:
            apha = tem0
        if tem0 < beta:
            beta = tem0
        chessboard[i][j] = 2
        if beta <= apha:
            return tem0, (i, j)

        if type == 1:
            return max(back_score_list), (score_list[back_score_list.index(max(back_score_list))][1], \
                   score_list[back_score_list.index(max(back_score_list))][2])
        if type == -1:
            return min(back_score_list), (score_list[back_score_list.index(min(back_score_list))][1], \
                   score_list[back_score_list.index(min(back_score_list))][2])


def play(c1,c2,generation):
    board_size = 15
    chessboard = torch.zeros((15,15))+2
    c1_turn =(8,8)
    boundary=[-1]*4
    while 1:
        chessboard[c1_turn[0]][c1_turn[1]]=1
        boundary = update_boundary(boundary, c1_turn[0], c1_turn[1], board_size)
        if winner_check(chessboard,c1_turn,board_size,1)==1:
            break
        if torch.max(chessboard) < 2:
            return 1
            #平局
            #score 是不同的
        score2,c2_turn=tree_search(chessboard,c2,boundary,(0, 0),2,-1, -100000000, 100000000)
        chessboard[c2_turn[0]][c2_turn[1]] = -1
        boundary = update_boundary(boundary, c2_turn[0], c2_turn[1], board_size)
        if winner_check(chessboard,c2_turn,board_size,-1)==1:
            break
        score1,c1_turn=tree_search(chessboard,c1,boundary,(0, 0),2,1, -100000000, 100000000)
    draw_check(screen)
    for i in range(board_size):
        for j in range(board_size):
            draw_checker(screen, i , j , chessboard[i][j])


    pygame.display.update()
    pygame.display.set_caption("N0.{} match".format(generation))
    if winner_check(chessboard,c2_turn,board_size,-1) == 1:
        print("white win")
        return 1
    else:
        print("black win")
        return 0

def competation(population,generation):
    l =len(population)#len应为双数promoted[i],promoted[i+1]
    i=0
    promoted=[]
    print(l)
    while i<l:
        x=population[i]
        f =play(population[i],population[i+1],generation)
        promoted.append(population[i+f])
        i+=2
    return promoted
#还要先交配 再 选出其余一半子代


def reproduction(p,q):
    x = random.randint(0,p.fc1.weight.shape[0]*p.fc1.weight.shape[1])
    y = random.randint(0, p.fc1.weight.shape[0] * p.fc1.weight.shape[1])
    a=p.fc1.weight
    b=p.fc2.weight
    p.fc1.weight[0,x:y]=q.fc1.weight[0,x:y]
    q.fc1.weight[0,x:y]=a[0,x:y]
    x = random.randint(0, p.fc2.weight.shape[0] * p.fc2.weight.shape[1])
    y = random.randint(0, p.fc2.weight.shape[0] * p.fc2.weight.shape[1])
    p.fc2.weight[0,x:y]=q.fc2.weight[0,x:y]
    q.fc2.weight[0,x:y]=b[0,x:y]
    #print(q.fc2.weight[0,x:y])
    return p,q






#how to initialize stochastic net
def genetic_algrothim(population,generation):#weight_tensor is object list
    if generation==0:
        torch.save(population[0],"net_params.pkl")
        print("ok")
        return population#to get the best one

    mutation_rate =0.3
    # how to initialize stochastic net

    promoted =competation(population,generation)#needs object
    population += promoted
    l =len(promoted)
    print(len(promoted))

    for i in range(len(promoted),1):
        promoted[i],promoted[i+1]=reproduction(promoted[i],promoted[i+1])
    for i in range(l):
        promoted.append(population[random.randint(0, len(population)-1)])
    #mutation
    for i in promoted:
        for j in range(i.fc1.weight.shape[1]):
            if random.random() < mutation_rate:
                i.fc1.weight[0,j] += random.random()
        for j in range(i.fc2.weight.shape[1]):
            if random.random() < mutation_rate:
                i.fc2.weight[0,j] += random.random()
    torch.save(population[0], "net_params0.pkl")
    torch.save(population[1], "net_params1.pkl")
    torch.save(population[2], "net_params2.pkl")
    torch.save(population[3], "net_params3.pkl")
    answer=genetic_algrothim(promoted,generation-1)
    return answer
#
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
a = torch.zeros(225)

population,weight=[],[]
for i in range(100):
    people = F_c(225,10,1)
    torch.manual_seed(i)
    torch.nn.init.normal_(people.fc1.weight,mean=0,std=1)
    torch.nn.init.normal_(people.fc2.weight,mean=0,std=1)
    people.to(device)
    population.append(people)


#weight=torch.tensor(weight)
while True:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            pygame.quit()

        print(weight)
        new_weight =genetic_algrothim(population,10000)

        print(new_weight)

 转载请注明出处！