【语音控制ROS】PocketPhinx语音包的使用＜三＞

2023-05-16

语音控制机器人

【语音控制ROS】虚拟机安装Ubuntu14.04+ROS-indigo＜一＞
【语音控制ROS】仿真环境的搭建＜二＞

用语音来控制机器人移动

前提：安装好了ROS环境 14.04+indigo

1、安装PocketSphinx语音识别

安装如下环境

sudo apt-get install gstreamer0.10-pocketsphinx
sudo apt-get install ros-indigo-pocketsphinx
sudo apt-get install ros-indigo-audio-common
sudo apt-get install libasound2

2、测试 PocketSphinx的语音功能

最好有个usb的麦克风，并且测试是否有声音输入

pocketsphinx 是ROS Indigo中的一个功能包. 更多的细节在ROS WIKIpocketsphinx package .
进入 pocketsphinx 功能包

roscd pocketsphinx

在目录 demo/, 我们能找到 voice commands and dictionary，可以用more/cat/nano查看

接下来启动robocup.launch文件

roslaunch pocketsphinx robocup.launch

文件robocup.launch如下

<launch>
  <node name="recognizer" pkg="pocketsphinx" type="recognizer.py" output="screen">
    <param name="lm" value="$(find pocketsphinx)/demo/robocup.lm"/>
    <param name="dict" value="$(find pocketsphinx)/demo/robocup.dic"/>
  </node>
</launch>

启动了 recognizer.py 脚本来运行 robocup.dic. 加载语音识别模型。启动成功为如下界面

INFO: ngram_search_fwdtree.c(186): Creating search tree
INFO: ngram_search_fwdtree.c(191): before: 0 root, 0 non-root channels, 12 single-phone words
INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 328
INFO: ngram_search_fwdtree.c(338): after: 77 root, 200 non-root channels, 11 single-phone words

接下来，对着麦克风说话

hello
go to the room
my name is
door
follow room

我们说的话是话题 /recognizer/output

rostopic echo /recognizer/output

可以看到输出的效果

data: go to the room
--
data: hello
--

3、创建工作区

创建工作区~/catkin_ws/src/
创建功能包 voice_robot 并且添加依赖 pocketsphinx, roscpp, rospy, sound_play and std_msgs

catkin_create_pkg voice_robot roscpp rospy pocketsphinx sound_play std_msgs

$ cd ~/catkin_ws
~/catkin_ws & catkin_make

4、创建语音命令

添加“ PocketSphinx”中指定的词汇或语料库。创建一个简单的命令词汇表，以使Turtlebot机器人向前，向后移动以及向左和向右旋转。

4.1 voice_robot下创建目录`config`

创建文件 motion_commands.txt 输入以下内容

move forward
move backwards
turn right
turn left

创建文件motion_commands.lm输入以下内容

Language model created by QuickLM on Wed Jun 15 13:21:57 EDT 2016
Copyright (c) 1996-2010 Carnegie Mellon University and Alexander I. Rudnicky

The model is in standard ARPA format, designed by Doug Paul while he was at MITRE.

The code that was used to produce this language model is available in Open Source.
Please visit http://www.speech.cs.cmu.edu/tools/ for more information

The (fixed) discount mass is 0.5. The backoffs are computed using the ratio method.
This model based on a corpus of 5 sentences and 9 words

\data\
ngram 1=9
ngram 2=12
ngram 3=9

\1-grams:
-0.8808 </s> -0.3010
-0.8808 <s> -0.2398
-1.5798 BACKWARD -0.2398
-1.5798 FORWARD -0.2398
-1.5798 LEFT -0.2398
-1.2788 MOVE -0.2775
-1.5798 RIGHT -0.2398
-1.5798 STOP -0.2398
-1.2788 TURN -0.2775

\2-grams:
-0.6990 <s> MOVE 0.0000
-1.0000 <s> STOP 0.0000
-0.6990 <s> TURN 0.0000
-0.3010 BACKWARD </s> -0.3010
-0.3010 FORWARD </s> -0.3010
-0.3010 LEFT </s> -0.3010
-0.6021 MOVE BACKWARD 0.0000
-0.6021 MOVE FORWARD 0.0000
-0.3010 RIGHT </s> -0.3010
-0.3010 STOP </s> -0.3010
-0.6021 TURN LEFT 0.0000
-0.6021 TURN RIGHT 0.0000

\3-grams:
-0.6021 <s> MOVE BACKWARD
-0.6021 <s> MOVE FORWARD
-0.3010 <s> STOP </s>
-0.6021 <s> TURN LEFT
-0.6021 <s> TURN RIGHT
-0.3010 MOVE BACKWARD </s>
-0.3010 MOVE FORWARD </s>
-0.3010 TURN LEFT </s>
-0.3010 TURN RIGHT </s>

\end\

创建文件motion_commands.dic并输入以下内容

BACKWARD	B AE K W ER D
FORWARD	F AO R W ER D
LEFT	L EH F T
MOVE	M UW V
RIGHT	R AY T
STOP	S T AA P
TURN	T ER N

4.2 voice_robot下创建`launch`目录

并创建 recognizer.launch

 <launch>
     <node name="recognizer" pkg="pocketsphinx" type="recognizer.py" output="screen">
       <param name="lm" value="$(find voice_robot)/config/motion_commands.lm"/>
       <param name="dict" value="$(find voice_robot)/config/motion_commands.dic"/>
     </node>
</launch>

启动launch文件 recognizer.launch

roslaunch voice_robot recognizer.launch

对着麦克风说话，观察输出

rostopic echo /recognizer/output

正常输出，接着下一步，控制机器人

5、Voice-Control的脚本

5.1 创建`script`目录

创建 voice_teleop.py

#!/usr/bin/env python

import rospy
from geometry_msgs.msg import Twist
from std_msgs.msg import String

class RobotVoiceTeleop:
    #define the constructor of the class
    def  __init__(self):
        #initialize the ROS node with a name voice_teleop
        rospy.init_node('voice_teleop')

        # Publish the Twist message to the cmd_vel topic
        self.cmd_vel_pub = rospy.Publisher('/cmd_vel', Twist, queue_size=5)

        # Subscribe to the /recognizer/output topic to receive voice commands.
        rospy.Subscriber('/recognizer/output', String, self.voice_command_callback)

        #create a Rate object to sleep the process at 5 Hz
        rate = rospy.Rate(5)

        # Initialize the Twist message we will publish.
        self.cmd_vel = Twist()
        #make sure to make the robot stop by default
        self.cmd_vel.linear.x=0;
        self.cmd_vel.angular.z=0;



        # A mapping from keywords or phrases to commands
        #we consider the following simple commands, which you can extend on your own
        self.commands =             ['stop',
                                'forward',
                                'backward',
                                'turn left',
                                'turn right',
                                ]
        rospy.loginfo("Ready to receive voice commands")
        # We have to keep publishing the cmd_vel message if we want the robot to keep moving.
        while not rospy.is_shutdown():
            self.cmd_vel_pub.publish(self.cmd_vel)
            rate.sleep()


    def voice_command_callback(self, msg):
        # Get the motion command from the recognized phrase
        command = msg.data
        if (command in self.commands):
            if command == 'forward':
                self.cmd_vel.linear.x = 0.2
                self.cmd_vel.angular.z = 0.0
            elif command == 'backward':
                self.cmd_vel.linear.x = -0.2
                self.cmd_vel.angular.z = 0.0
            elif command == 'turn left':
                self.cmd_vel.linear.x = 0.0
                self.cmd_vel.angular.z = 0.5
            elif command == 'turn right':
                self.cmd_vel.linear.x = 0.0
                self.cmd_vel.angular.z = -0.5
            elif command == 'stop':
                self.cmd_vel.linear.x = 0.0
                self.cmd_vel.angular.z = 0.0

        else: #command not found
            #print 'command not found: '+command
            self.cmd_vel.linear.x = 0.0
            self.cmd_vel.angular.z = 0.0
        print ("linear speed : " + str(self.cmd_vel.linear.x))
        print ("angular speed: " + str(self.cmd_vel.angular.z))



if __name__=="__main__":
    try:
      RobotVoiceTeleop()
      rospy.spin()
    except rospy.ROSInterruptException:
      rospy.loginfo("Voice navigation terminated.")

新建launch文件voice_control_in_stage.launch

 <launch>
    <node name="recognizer" pkg="pocketsphinx" type="recognizer.py" output="screen">
       <param name="lm" value="$(find voice_robot)/config/motion_commands.lm"/>
       <param name="dict" value="$(find voice_robot)config/motion_commands.dic"/>
   </node>

   <node name="voice_teleop" pkg="voice_robot" type="voice_teleop.py" output="screen">
      <remap from="/cmd_vel" to="/cmd_vel_mux/input/teleop"/>
   </node>

   <include file="$(find turtlebot_stage)/launch/turtlebot_in_stage.launch"/>
</launch>

使用gazebo仿真的话，最后一句换为以下代码

<launch>
...
  <include file="$(find turtlebot_gazebo)/launch/turtlebot_world.launch"/>
</launch>

6 测试

启动launch文件

roslaunch voice_robot voice_control_in_stage.launch

相当于下面三句话

roslaunch voice_robot recognizer.launch
rosrun voice_robot voice_teleop.py
roslaunch turtlebot_stage turtlebot_in_stage.launch

对着麦克风说话测试成功

7 可继续做的工作

7.1 扩展词汇

7.2 语音决定机器人是否接收语音

本文内容由网友自发贡献，版权归原作者所有，本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容，请联系:hwhale#tublm.com(使用前将#替换为@)

ROS

PocketPhinx

语音控制

语音包的使用