Build your First Voice Bot with Rasa <3

Voice… Voice… Voice… 🔊

As we all are using the technology for automating the resources, we are also advancing the way it will do the automation. Like till now we were using the chatbots for automation but as we are advancing the way of implementation, our chatbots are also getting converted to the voice bots. I am back with this advancing technique for you to apply it in real time.


In this post you will learn how to build your first voice bot by integrating it with Rasa chatbot. The integration of rasa chatbot for it conversion to a voice bot have a journey that starts from voice and ends with voice that makes it user friendly and user interactive. Steps that are included here for voice bot are:

1. Speech to text conversion

2. Externally sending an input to rasa chatbot

3. Text to speech conversion

Steps mentioned above are the most important one that will help you make an effective and user interactive bot. In the first step, the user will give an input message in the form of speech that is supposed to be to be sent to the rasa chatbot so that it can give response as per the phrase said. But rasa chatbot only accepts the textual input so we need to convert speech to text and then to further send it to rasa chatbot.

In the second step, the converted text is passed to the rasa chatbot so that your trained rasa chatbot will respond properly as per the stories created. But here the input is not send directly by simply running rasa x or rasa shell instead we need to setup few things and we will use a seperate command that will run rasa server in backend and using this server your chabtbot will be able to respond according and you will get the output as the textual data.

Now, when you will get this textual output from rasa chatbot you need to now convert it back to the speech as the user has provide the input in the form of speech so for this step to make it more interactive and attractive we will use this text and convert it to speech.

If you are new here and python programming is new for you then check this link and get started with python and start your journey from Zero to Hero.

Build your First Voice Bot with Rasa

Now let’s start one by one and understand how to do it.

Step 1: Speech to Text

Speech to text conversion is the process of converting spoken words into written texts. This process is also often called speech recognition. Although these terms are almost synonymous, Speech recognition is sometimes used to describe the wider process of extracting meaning from speech, i.e. speech understanding. The term voice recognition should be avoided as it is often associated to the process of identifying a person from their voice, i.e. speaker recognition.

In this step, we will ask the user to say a message and that message will be recording using the microphone connected to your system and the this recording audio will be send to the object created using SpeechRecognition module that we will use here for speech to text conversion.

The python module SpeechRecognition and PyAudio are used here where SpeechRecognition is created by google, which makes it easier for speech to text conversion.First install the required module for the conversion by typing the following command in the terminal and installing it in the virtual environment,

$ pip install SpeechRecognition PyAudio

Here is the program for this conversion:

import speech_recognition as sr     # import the library
r = sr.Recognizer() # initialize recognizer
with sr.Microphone() as source: # mention source it will be either Microphone or audio files.
print("Speak Anything :")
audio = r.listen(source) # listen to the source
text = r.recognize_google(audio) # use recognizer to convert our audio into text part.
print("You said : {}".format(text))
print("Sorry could not recognize your voice") # In case of voice not recognized clearly

To understand it with practical implementation check the video,

Step 2: Externally sending an input to rasa chatbot

In this step, we will use our trained rasa chatbot that we have already built in the previous sessions and we will setup few easy things that are required for calling the rasa chatbot externally and to send an input message t it that we have received in step 1 and futher the chatbot will reply you accordingly. Here we need to make changes in the credentials file, endpoints file and to call it seperately to call rasa chatbot in backend.

If this is your first post that you are reading on my website and if you have skipped how to build your own rasa chatbot the do checkout this playlist and get the understanding of Rasa chatbot to become and expert in this field,

In credentials file, add these lines and uncomment it if those are commented:

## you don't need to provide anything here

url: "http://localhost:5002/api"
In endpoints file,
url: "http://localhost:5055/webhook"

When these files has been setup then run the following command in your project directory in terminal,

$ rasa run -m models --endpoints endpoints.yml --port 5002 --credentials credentials.yml

This will start your rasa chatbot in backend now add the following lines of code to send the received text (from step 1) to rasa chatbot externally and to get the corresponding out for it.

import requests
sender = input("What is your name?\n")
bot_message = ""
while bot_message != "Bye":
message = input("What's your message?\n")
print("Sending message now...")
r ='http://localhost:5002/webhooks/rest/webhook', json={"sender": sender, "message": message})
print("Bot says, ",end=' ')
for i in r.json():
bot_message = i['text']

after the execution of this code you will get the output from rasa chatbot(in textual form) as per your trained model. Now make it more interactive and attractive by converting the output to speech again. To understand it with practical implementation check the video

Step 3: Text to speech conversion
Text to speech, abbreviated as TTS, is a form of speech synthesis that converts text into spoken voice output. Text to speech systems were first developed to aid the visually impaired by offering a computer-generated spoken voice that would “read” text to the user.

In this step, we will learn how to convert the text to speech using the most useful and easiest way of conversion using gTTS module that is google Text to Speech which is designed by google for the needy one like the blind person or those who can’t view properly. For this we will use the gTTs class that is use to convert the text to speech and we will further save it to any of the audio format and then we will play it with the inbuilt module named subprocess that will help play and exit the audio using any of the installed media player.

Firstly we will install the gTTs module by simple typing the following command in the terminal and to install it in the virtual environment.

$ pip install gtts

and also install the media player for playing the audio after conversion,

$ sudo apt-get install mpg321

When everything is installed as per the requirements the add the following code to make it happen,

import subprocess
from gtts import gTTS
# The text that you want to convert to audio
mytext = 'Welcome to Innovate Yourself!'
# Language in which you want to convert
language = 'en'

myobj = gTTS(text=mytext, lang=language)
# Saving the converted audio in a mp3 file named
# welcome"welcome.mp3")
# Playing the converted file['mpg321', "welcome.mp3", '--play-and-exit'])

When everything is working fine we need to combine the program of all three steps and to make an effective voice bot that can handle automation. To understand it with practical implementation check the video

To get the full code of the working of this chatbot then check this link:

Rasa chatbot Customer Service Full Code

Now to see your voice bot in action run the file and also run the action server using

$ rasa run actions

Note: Before testing your bot make sure you have an active internet connection on your system. Now, You can see your Voice bot in action. To get the full code understanding and to learn how to merge the code to see the final working and it’s practical implementation check this video,

Time to wrap up now. Hope you liked our content on How to build your first voice bot with Rasa chatbot. See you in our next blog, thanks for reading our blog and do leave a comment below to help us improve the content to serve you all of our experience with you. Stay tuned with us for more Rasa Chatbot contents.

Also check out our other playlist Rasa Chatbot, Internet of things, Docker, Python Programming, etc.
Become a member of our social family on youtube here.

Ashish Saini
Ashish Saini

I am a software developer for Python/Machine Learning. Also, I’m Rasa Hero at Rasa. Also, I’m a youtuber where I regulary uploads the videos to help this generation learn about the technology just like they are playing games.

5 thoughts on “Build your First Voice Bot with Rasa <3

  1. This audio bot is perfect for the localhost. Is there any way to use client-side webrtc , so that it can be used as a source instead of a microphone?

  2. This audio bot is perfect for the localhost. Is there any way to use client-side webrtc , so that it can be used as a source instead of a microphone?

  3. why it is showing the following error when I am trying to run speech to text

    Traceback (most recent call last):
    File “f:/voicebot_demo/”, line 24, in[‘mpyg321’, “welcome.mp3”, ‘–play-and-exit’])
    File “C:\anaconda3\envs\project_rasa\lib\”, line 339, in call
    with Popen(*popenargs, **kwargs) as p:
    File “C:\anaconda3\envs\project_rasa\lib\”, line 800, in __init__
    restore_signals, start_new_session)
    File “C:\\anaconda3\envs\project_rasa\lib\”, line 1207, in _execute_child
    FileNotFoundError: [WinError 2] The system cannot find the file specified

Leave a Reply

Your email address will not be published. Required fields are marked *