OVERVIEW
In this blog you will learn how to build an end-user chatbot that you can share with your friends/collegues/family members to help them as an assistant. But how do we do it? Don’t worry in this blog you will learn everything about it from scratch, means from Zero to the Hero level.
In this blog you will learn,
- Setup and Installation of rasa chatbot
- How to create intents
- How to create stories
- How to extract the entities with and without Duckling server
- How to use and create forms
- How to store the data to the excel sheet
- How to extract the information from the excel sheet and display in the output
So, let’s begin with the first step.
Setup and Installation of rasa chatbot
There are dependencies and different packages that are required to be installed in an environment or docker container before starting with the rasa chatbot project. I have already covered this in one of my previous blogs, Build an end-to-end Conversational chatbot with UI. You can go through this for the proper installation.
Once, you have all the packages installed on to your environment the initialize the project with,
rasa init –no-prompt
Also, check this if you have any doubts while installation,
This will create a basic project for you on your current directory.
Once you will have the basic model trained, you can talk to the chatbot with rasa shell or rasa x and test it’s working. Now, we can proceed and understand how to read and store the data to the excel sheet.
Before proceeding further lets understand what are we expecting from the chatbot. So, from our chatbot we are expecting,
- Collect the user data and store in an excel(.xlsx) sheet.
- Based on user input extract the information that a user is asking for.
To understand this scenario I will add 2 conversations to the chatbot with, one for collecting the data and storing in to the excel sheet and second to extract the data based on user input.
Conversation 1: User: i want to add some data to the excel sheet Bot: May I Know your name please User: Ashish Bot: May I Know your mobile number please User: 8209829808 Bot: May I Know your email id please User: [email protected] Bot: May I Know your occupation please User: Software Developer Bot: Here are the information that you provided. Do you want to save it? Name: Ashish, Mobile Number: 8209829808, Email: [email protected], Occupation: Software Developer User: yes Bot: Data Stored Successfully.
Conversation 2: User: tell me the email of the user whose occupation is software developer Bot: This is the data that you asked for, [email protected]
So, here you can see two different conversations. Now let’s start with the first conversation for collecting the information from the user and storing to the excel sheet. Let’s start by creating the intents first.
How to create intents
Add the given intent data to the nlu.md file for asking the chatbot to add the data to the excel sheet,
nlu.md
## intent:add_data - i want to add some data to the excel sheet - i want to add the user data to the excel sheet - add user data - ask me some questions to add the user data - what information is required to add the data to the excel sheet
Now add the intent name add_data to the domain.yml file to register it,
domain.yml
intents:
- add_data
So, the intent is registered to the domain file, further as per the conversation our chatbot should ask for the multiple questions to collect and store it to the slots and to that step I’ll use the most effective and the convenient method known as Forms which will collect information from the user and will autofill them to the slots. You can check this video if you have no idea what forms are,
So, forms works based on the slots as we set the prefrences and order of the slots based on that the chatbot will ask you the questions to full the slot value. Which means you must have the slots already set in the domain file as per the information that you want to collect. In our case I’ll be collecting name, mobile number, email and occupation. For that create a seperate slot in the domain file in this way,
domain.yml
slots: name: type: unfeaturized number: type: unfeaturized email: type: unfeaturized occupation: type: unfeaturized
This will set the slots for storing the collecting information. Based on that you also have to add the responses for the chatbot in domain file. As, these are the utter actions for forms so the name of the response will be like this utter_ask_, for that just add the given responses into the domain file,
domain.yml
responses: utter_ask_name: - text: "May I Know your name please" utter_ask_number: - text: "May I Know your mobile number please" utter_ask_email: - text: "May I Know your email id please" utter_ask_occupation: - text: "May I Know your occupation please"
How to extract the entities with Duckling server
Here, name and occupation can be anything so for our case I’ll be picking up the direct value from the user and will store it in the slots. But for the mobile number and email i want to extract the values from the user input in the form of entities. So, for the entity extraction of email and mobile number I’ll use the duckling which is capable enough to extract multiple data so as the number and email. For that I have to add the enties to the domain file as,
entities: - number - email
To understand it better with proper functionality check this video,
This will register the entities to the domain file so that you can further use it to read and store the entities value.
Now, as we have already setup the slot names and the responses for it. It’s time to add the form name to the domain file and to create the form in the action file. Add the form name here,
forms:
- Form_Info
How to use and create forms
Now add this form action to the actions.py script to link the functionality to the data registered to the domain file.
actions.py
class FormDataCollect(FormAction):
def name(self) -> Text:
return "Form_Info"
@staticmethod
def required_slots(tracker: "Tracker") -> List[Text]:
return ["name","number","email","occupation"]
def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict[Text, Any]]]]:
return {
"name":[self.from_text()],
"number":[self.from_entity(entity="number")],
"email":[self.from_entity(entity="email")],
"occupation":[self.from_text()]
}
def submit(
self,
dispatcher: "CollectingDispatcher",
tracker: "Tracker",
domain: Dict[Text, Any],
) -> List[EventType]:
dispatcher.utter_message("Here are the information that you provided. Do you want to save it?\nName: {0},\nMobile Number: {1},\nEmail: {2},\nOccupation: {3}".format(
tracker.get_slot("name"),
tracker.get_slot("number"),
tracker.get_slot("email"),
tracker.get_slot("occupation"),
))
return []
Here, this form action will be linked to the different slots added in the domain.yml file and to further fill the extracted data to the slots in the order that is set in here,
@staticmethod
def required_slots(tracker: "Tracker") -> List[Text]:
return ["name","number","email","occupation"]
This means the questions will be asked in this order from the name to the occupation and also these will be auto filled based on the slot mappings done here,
def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict[Text, Any]]]]:
return {
"name":[self.from_text()],
"number":[self.from_entity(entity="number")],
"email":[self.from_entity(entity="email")],
"occupation":[self.from_text()]
}
Once, all the slots are filled with the information extracted from the user input the Submit method will be called as shown here,
def submit(
self,
dispatcher: "CollectingDispatcher",
tracker: "Tracker",
domain: Dict[Text, Any],
) -> List[EventType]:
dispatcher.utter_message("Here are the information that you provided. Do you want to save it?\nName: {0},\nMobile Number: {1},\nEmail: {2},\nOccupation: {3}".format(
tracker.get_slot("name"),
tracker.get_slot("number"),
tracker.get_slot("email"),
tracker.get_slot("occupation"),
))
return []
This method will ask you for the affirmation to store the data to the excel sheet.
How to store the data to the excel sheet
For storing the data to the excel sheet I’ll write a separate script where I’ll create a function and link it to the rasa chatbot to store the collected information to excel sheet.
To store the data I will use the pandas which are really helpful and useful for storing and reading the data (statistical analysis). So, I’ve to install it first before using it. To install it run the below command to in the terminal by activating the rasa environment,
pip install pandas
This will install pandas in your environment and now you can create a seperate function in the seperate script for storing the data.
Create a seperate script and add the given function in to it.
import pandas as pd import os def DataStore(name,mobile_number,email,occupation): if os.path.isfile("user_data.xlsx"): df=pd.read_excel("user_data.xlsx") df=df.append(pd.DataFrame([[name,mobile_number,email,occupation]], columns=["name","mobile_number","email","occupation"])) df.to_excel("user_data.xlsx",index=False) else: df=pd.DataFrame([[name,mobile_number,email,occupation]], columns=["name","mobile_number","email","occupation"]) df.to_excel("user_data.xlsx",index=False) return []
The above function will store the to the excel sheet if the file doesn’t exist and if it exists the it will update the data to the existing sheet.
Now let’s link this function to the form action that we created. To do this, first create an action in the actions.py script by adding the given code,
from excel_data_store_read import DataStore
class ActionSaveData(Action):
def name(self) -> Text:
return "action_save_data"
def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
DataStore(tracker.get_slot("name"),
tracker.get_slot("number"),
tracker.get_slot("email"),
tracker.get_slot("occupation"))
dispatcher.utter_message(text="Data Stored Successfully.")
return []
This function will store the data to the excel sheet with respect to the function created in the separate script.
How to create stories
Add the given story to the stories.md file so that these functions should work,
stories.md
## store data to excel affirm Story * add_data - Form_Info - form{"name": "Form_Info"} - form{"name": null} * affirm - action_save_data
So, we are now done with our first conversation. So, you can now train the mode and test the chatbot for the above story and you will see the data stored in the excel sheet. For this story the output will be something like this,
How to extract the information from the excel sheet and display in the output
Now, let’s update the chatbot as per the second conversation, and understand how to extract the information based on the user input.
Add the given data to nlu.md to accept what user says,
## intent:ask_info - list out the [email]{"entity":"column","value":"email"} of the user who's occupation is [trainer]{"entity":"occup","value":"trainer"} - list out the [name]{"entity":"column","value":"name"} of the user who's occupation is [doctor]{"entity":"occup","value":"doctor"} - tell me the [mobile number]{"entity":"column","value":"number"} of the user whose occupation is [software developer]{"entity":"occup","value":"software developer"}
this is intent with the entities that will be extracted from the user input based on which information will be extracted from the user’s existing excel sheet. Add the intent name and entities in the domain file to register them.
intents: - ask_info entities: - column - occup
Based on these entities extracted, create an action that will be called when the user will ask for the information to extract.
Add the given line of code in to the actions.py script to extract the data,
class ActionFetchData(Action):
def name(self) -> Text:
return "action_fetch_data"
def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
output=FetchData(tracker.latest_message['entities'][0]['value'],
tracker.latest_message['entities'][1]['value'])
dispatcher.utter_message(text="This is the data that you asked for, \n{}".format(",".join(output)))
return []
In the above run method, you will see a function is called which will be created in the other script to read the data from the excel sheet,
def FetchData(column,occupation):
if os.path.isfile("user_data.xlsx"):
df=pd.read_excel("user_data.xlsx")
data=df[column][df["occupation"]==occupation]
return data.to_list()
else:
return ["There is no data. "]
Once, you have created the function for reading the data as per the user input. Also, add the stories in the stories.md file so as to follow the path,
## fetch info * ask_info{"column":"email","occup":"trainer"} - action_fetch_data
Also add this action name to domain file to register it,
actions: - action_fetch_data
For full clarification you can also check this session,
You can also download the full code here,
DOWNLOAD FULL CODE HERE
Once everything is done. Train the model and test it with the following commands running is separate terminal,
rasa train && rasa x
rasa run actions
docker run -p 8000:8000 rasa/duckling
This is how you chatbot will respond once everything is working fine,
Also, check these blogs if you want your chatbot to run on Telegram or facebook messenger.
Now, I hope everything is understood and now you can apply this into your bussiness model to increase your reach.
Time to wrap up now. Hope you liked our content on How to build an end-user chatbot. See you in our next blog, thanks for reading our blog and do leave a comment below to help us improve the content to serve you all of our experience with you. Stay tuned with us for more Rasa Chatbot contents.
Also check out our other playlist Rasa Chatbot, Internet of things, Docker, Python Programming, etc.
Become a member of our social family on youtube here.
Hi this is a wonderful article.
I downloaded the code from GitHub and run it I am able to give details and data is storing as well. but when I am trying to retrieve the data from excel it’s showing me below error.
Exception occurred while handling uri: ‘http://localhost:5055/webhook’
Traceback (most recent call last):
File “/home/rahul/venv/lib/python3.7/site-packages/sanic/app.py”, line 976, in handle_request
response = await response
File “/home/rahul/venv/lib/python3.7/site-packages/rasa_sdk/endpoint.py”, line 102, in webhook
result = await executor.run(action_call)
File “/home/rahul/venv/lib/python3.7/site-packages/rasa_sdk/executor.py”, line 387, in run
events = action(dispatcher, tracker, domain)
File “/home/rahul/Downloads/RasaChatbotExcelData-master/actions.py”, line 45, in run
tracker.latest_message[‘entities’][1][‘value’])
IndexError: list index out of range
Your help is highly appreciated.
Thank you
Regards
Rahul
It’s because there is only one entity that is extracted and the code is written to extract and fetch 2 values.
got it. I am actually looking for fetching values that are stored in my sql. Can you please do a tutorial where we can fetch details from my sql or sql server.
Thank you
Hi, I’m happy that you found my blog useful for you. I’m your case you can check this link it will help you to fetch the data from mysql database
https://youtu.be/Q1znA4Svhew
Hi, I’m happy that you found my blog useful for you. I’m your case you can check this link it will help you to fetch the data from mysql database
https://youtu.be/Q1znA4Svhew
Hi, This is really helpful thanks a lot. Can you also make a video where we can have interactions with external API services?
You mean web scraping??
Hello sir i am getting error while using duckling can you please help me. I think my rasa/duckling is working as it gives output as : Listening on http://0.0.0.0:8000
Then I also made sure I did rasa run actions.
And when I finally do rasa shell, it works fine and then if I give any input it shows :
ERROR rasa.nlu.extractors.duckling_http_extractor – Failed to connect to duckling http server. Make sure the duckling server is running/healthy/not stale and the proper host and port are set in the configuration. More information on how to run the server can be found on github: https://github.com/facebook/duckling#quickstart Error: (‘Connection aborted.’, OSError(0, ‘Error’))
Hello sir, Thanks for giving wonderful article about chatbot formation. Its amazing work about chatbot.
Hello sir thanks for using information about chatbot services , this is from vajra.ai chatbot service related page.
Hello sir, Thanks for giving wonderful article about chatbot formation. Its amazing work about chatbot.
This is from https://vajra.ai/ is the related to chatbot.
No, for example within a weather bot (external API) where we can do CRUD operations via interactions. Can you make a video on these topics? it will be really helpful.
Thank you
Traceback (most recent call last):
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
domain.py”, line 546, in index_for_action
return self.action_names.index(action_name)
ValueError: None is not in list
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\runpy.py”, line 193, in
_run_module_as_main
“__main__”, mod_spec)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\runpy.py”, line 85, in _
run_code
exec(code, run_globals)
File “C:\Users\Sathish\anaconda3\envs\pythonProject12\Scripts\rasa.exe\__main__.py
“, line 7, in
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\__mai
n__.py”, line 92, in main
cmdline_arguments.func(cmdline_arguments)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\cli\t
rain.py”, line 76, in train
additional_arguments=extract_additional_arguments(args),
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
.py”, line 50, in train
additional_arguments=additional_arguments,
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\asyncio\base_events.py”,
line 587, in run_until_complete
return future.result()
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
.py”, line 101, in train_async
additional_arguments,
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
.py”, line 188, in _train_async_internal
additional_arguments=additional_arguments,
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
.py”, line 223, in _do_training
additional_arguments=additional_arguments,
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
.py”, line 361, in _train_core_with_validated_data
additional_arguments=additional_arguments,
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
train.py”, line 66, in train
agent.train(training_data, **additional_arguments)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
agent.py”, line 742, in train
self.policy_ensemble.train(training_trackers, self.domain, **kwargs)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
policies\ensemble.py”, line 124, in train
policy.train(training_trackers, domain, **kwargs)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
policies\memoization.py”, line 163, in train
self._add_states_to_lookup(trackers_as_states, trackers_as_actions, domain)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
policies\memoization.py”, line 110, in _add_states_to_lookup
feature_item = domain.index_for_action(action)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
domain.py”, line 548, in index_for_action
self._raise_action_not_found_exception(action_name)
File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
domain.py”, line 553, in _raise_action_not_found_exception
f”Cannot access action ‘{action_name}’, ”
NameError: Cannot access action ‘None’, as that name is not a registered action for
this domain. Available actions are:
– action_listen
– action_restart
– action_session_start
– action_default_fallback
– action_deactivate_form
– action_revert_fallback_events
– action_default_ask_affirmation
– action_default_ask_rephrase
– action_back
– utter_ask_email
– utter_ask_name
– utter_ask_number
– utter_goodbye
– utter_greet
– Form_Info
Check in the stories, may be there you forgot to add anything in the place of bot action.
Check in the stories, maybe there you forgot to add anything in the place of bot action.
Thanks for the amazing videos
I want to deploy the basic rasa chatbot on some website using flask
Just the basic one which come inbuilt with rasa.
Can you please guide me how can I do it
Sure tell me how can I help you
Hello sir i am getting error while using duckling can you please help me. I think my rasa/duckling is working as it gives output as : Listening on http://0.0.0.0:8000 Then I also made sure I did rasa run actions. And when I finally do rasa shell, it works fine and then if I give any input it shows : ERROR rasa.nlu.extractors.duckling_http_extractor – Failed to connect to duckling http server. Make sure the duckling server is running/healthy/not stale and the proper host and port are set in the configuration. More information on how to run the server can be found on github: https://github.com/facebook/duckling#quickstart Error: (‘Connection aborted.’, OSError(0, ‘Error’))
Are you running it on windows?
i am using linux sir
ubuntu.
Ok, so it means your duckling server is not connected properly to your chatbot. Check the pipeline again and correct it
i did as it was mentioned on tutorial and I am also getting listening on https://0.0.0.0:8000. Will we get that message even if docker is not running.
No you will not get it then. Surely your duckling is not connected properly to rasa
Hey,
Great Blog indeed, thank you for providing this content.
I want to ask that can these things possible by voice commands too. Have you made this thing, if yes then kindle provide the link or any other resources from where I can learn.
Regards
Yes, check the voice bot video on my youtube channel “INNOVATE YOURSELF”.
Hi, the chat which is created in your website solves my problem and I would like to recommend another website which provides chatbot services. Please visit https://vajra.ai/ to know more.