BUILD AN END-USER CHATBOT

In this blog you will learn how to build an end-user chatbot that you can share with your friends/collegues/family members to help them as an assistant. But how do we do it? Don’t worry in this blog you will learn everything about it from scratch, means from Zero to the Hero level.
In this blog you will learn,

  • Setup and Installation of rasa chatbot
  • How to create intents
  • How to create stories
  • How to extract the entities with and without Duckling server
  • How to use and create forms
  • How to store the data to the excel sheet
  • How to extract the information from the excel sheet and display in the output

So, let’s begin with the first step.

Setup and Installation of rasa chatbot

There are dependencies and different packages that are required to be installed in an environment or docker container before starting with the rasa chatbot project. I have already covered this in one of my previous blogs, Build an end-to-end Conversational bot with UI. You can go through this for the proper installation.
Once, you have all the packages installed on to your environment the initialize the project with,

rasa init –no-prompt

Also, check this if you have any doubts while installation,

https://youtu.be/GwaSJUlB8oA

This will create a basic project for you on your current directory.

Once you will have the basic model trained, you can talk to the bot with rasa shell or rasa x and test it’s working. Now, we can proceed and understand how to read and store the data to the excel sheet.
Before proceeding further lets understand what are we expecting from the chatbot. So, from this bot we are expecting,

  • Collect the user data and store in an excel(.xlsx) sheet.
  • Based on user input extract the information that a user is asking for.

To understand this scenario I will make the bot with 2 conversations, one for collecting the data and storing in to the excel sheet and second to extract the data based on user input.

Conversation 1:
User: i want to add some data to the excel sheet
Bot: May I Know your name please
User: Ashish
Bot: May I Know your mobile number please
User: 8209829808
Bot: May I Know your email id please
User: [email protected]
Bot: May I Know your occupation please
User: Software Developer
Bot: Here are the information that you provided. Do you want to save it?
Name: Ashish,
Mobile Number: 8209829808,
Email: [email protected],
Occupation: Software Developer
User: yes
Bot: Data Stored Successfully.
Conversation 2:
User: tell me the email of the user whose occupation is software developer
Bot: This is the data that you asked for,
[email protected]

So, here you can see two different conversations. Now let’s start with the first conversation for collecting the information from the user and storing to the excel sheet. Let’s start by creating the intents first.

How to create intents


Add the given intent data to the nlu.md file for asking the bot to add the data to the excel sheet,

nlu.md

## intent:add_data
- i want to add some data to the excel sheet
- i want to add the user data to the excel sheet
- add user data
- ask me some questions to add the user data
- what information is required to add the data to the excel sheet

Now add the intent name add_data to the domain.yml file to register it,

domain.yml

intents:
- add_data

So, the intent is registered to the domain file, further as per the conversation our bot should ask for the multiple questions to collect and store it to the slots and to that step I’ll use the most effective and the convenient method known as Forms which will collect information from the user and will autofill them to the slots. You can check this video if you have no idea what forms are,

So, forms works based on the slots as we set the prefrences and order of the slots based on that the bot will ask you the questions to full the slot value. Which means you must have the slots already set in the domain file as per the information that you want to collect. In our case I’ll be collecting name, mobile number, email and occupation. For that create a seperate slot in the domain file in this way,

domain.yml

slots:
name:
  type: unfeaturized
number:
  type: unfeaturized
email:
  type: unfeaturized
occupation:
  type: unfeaturized

This will set the slots for storing the collecting information. Based on that you also have to add the responses for the bot in domain file. As, these are the utter actions for forms so the name of the response will be like this utter_ask_, for that just add the given responses into the domain file,

domain.yml

responses:
utter_ask_name:
- text: "May I Know your name please"
utter_ask_number:
- text: "May I Know your mobile number please"
utter_ask_email:
- text: "May I Know your email id please"
utter_ask_occupation:
- text: "May I Know your occupation please"

How to extract the entities with Duckling server
Here, name and occupation can be anything so for our case I’ll be picking up the direct value from the user and will store it in the slots. But for the mobile number and email i want to extract the values from the user input in the form of entities. So, for the entity extraction of email and mobile number I’ll use the duckling which is capable enough to extract multiple data so as the number and email. For that I have to add the enties to the domain file as,

entities:
- number
- email

To understand it better with proper functionality check this video,

This will register the entities to the domain file so that you can further use it to read and store the entities value.

Now, as we have already setup the slot names and the responses for it. It’s time to add the form name to the domain file and to create the form in the action file. Add the form name here,

forms:
- Form_Info

How to use and create forms
Now add this form action to the actions.py script to link the functionality to the data registered to the domain file.

actions.py

class FormDataCollect(FormAction):
    def name(self) -> Text:
        return "Form_Info"
    @staticmethod
    def required_slots(tracker: "Tracker") -> List[Text]:
        return ["name","number","email","occupation"]

    def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict[Text, Any]]]]:
        return {
        "name":[self.from_text()],
        "number":[self.from_entity(entity="number")],
        "email":[self.from_entity(entity="email")],
        "occupation":[self.from_text()]
        }

    def submit(
        self,
        dispatcher: "CollectingDispatcher",
        tracker: "Tracker",
        domain: Dict[Text, Any],
    ) -> List[EventType]:

        dispatcher.utter_message("Here are the information that you provided. Do you want to save it?\nName: {0},\nMobile Number: {1},\nEmail: {2},\nOccupation: {3}".format(
        tracker.get_slot("name"),
        tracker.get_slot("number"),
        tracker.get_slot("email"),
        tracker.get_slot("occupation"),

        ))
        return []

Here, this form action will be linked to the different slots added in the domain.yml file and to further fill the extracted data to the slots in the order that is set in here,

@staticmethod
def required_slots(tracker: "Tracker") -> List[Text]:
return ["name","number","email","occupation"]

This means the questions will be asked in this order from the name to the occupation and also these will be auto filled based on the slot mappings done here,

def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict[Text, Any]]]]:
    return {
        "name":[self.from_text()],
        "number":[self.from_entity(entity="number")],
        "email":[self.from_entity(entity="email")],
        "occupation":[self.from_text()]
    }

Once, all the slots are filled with the information extracted from the user input the Submit method will be called as shown here,

def submit(
self,
dispatcher: "CollectingDispatcher",
tracker: "Tracker",
domain: Dict[Text, Any],
) -> List[EventType]:    
dispatcher.utter_message("Here are the information that you provided. Do you want to save it?\nName: {0},\nMobile Number: {1},\nEmail: {2},\nOccupation: {3}".format(
        tracker.get_slot("name"),
        tracker.get_slot("number"),
        tracker.get_slot("email"),
        tracker.get_slot("occupation"),

    ))
    return []

This method will ask you for the affirmation to store the data to the excel sheet.

How to store the data to the excel sheet
For storing the data to the excel sheet I’ll write a separate script where I’ll create a function and link it to the rasa chatbot to store the collected information to excel sheet.
To store the data I will use the pandas which are really helpful and useful for storing and reading the data (statistical analysis). So, I’ve to install it first before using it. To install it run the below command to in the terminal by activating the rasa environment,

pip install pandas

This will install pandas in your environment and now you can create a seperate function in the seperate script for storing the data.
Create a seperate script and add the given function in to it.

import pandas as pd
import os
def DataStore(name,mobile_number,email,occupation):
    if os.path.isfile("user_data.xlsx"):
        df=pd.read_excel("user_data.xlsx")
        df=df.append(pd.DataFrame([[name,mobile_number,email,occupation]],
            columns=["name","mobile_number","email","occupation"]))
        df.to_excel("user_data.xlsx",index=False)
    else:
        df=pd.DataFrame([[name,mobile_number,email,occupation]],
        columns=["name","mobile_number","email","occupation"])
        df.to_excel("user_data.xlsx",index=False)
        return []

The above function will store the to the excel sheet if the file doesn’t exist and if it exists the it will update the data to the existing sheet.

Now let’s link this function to the form action that we created. To do this, first create an action in the actions.py script by adding the given code,

from excel_data_store_read import DataStore
class ActionSaveData(Action):
def name(self) -> Text:
    return "action_save_data"

def run(self, dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
    DataStore(tracker.get_slot("name"),
        tracker.get_slot("number"),
        tracker.get_slot("email"),
        tracker.get_slot("occupation"))
    dispatcher.utter_message(text="Data Stored Successfully.")

    return []

This function will store the data to the excel sheet with respect to the function created in the seperate script.

How to create stories
Add the given story to the stories.md file so that these functions should work,

stories.md

## store data to excel affirm Story
* add_data
  - Form_Info
  - form{"name": "Form_Info"}
  - form{"name": null}
* affirm
  - action_save_data

So, we are now done with our first conversation. So, you can now train the mode and test the bot for the above story and you will see the data stored in the excel sheet. For this story the output will be something like this,

How to extract the information from the excel sheet and display in the output
Now, let’s update the bot as per the second conversation, and understand how to extract the information based on the user input.

Add the given data to nlu.md to accept what user says,

## intent:ask_info
- list out the [email]{"entity":"column","value":"email"} of the user who's occupation is [trainer]{"entity":"occup","value":"trainer"}
- list out the [name]{"entity":"column","value":"name"} of the user who's occupation is [doctor]{"entity":"occup","value":"doctor"}
- tell me the [mobile number]{"entity":"column","value":"number"} of the user whose occupation is [software developer]{"entity":"occup","value":"software developer"}

this is intent with the entities that will be extracted from the user input based on which information will be extracted from the user’s existing excel sheet. Add the intent name and entities in the domain file to register them.

intents:
- ask_info
entities:
- column
- occup

Based on these entities extracted, create an action that will be called when the user will ask for the information to extract.
Add the given line of code in to the actions.py script to extract the data,

class ActionFetchData(Action):
def name(self) -> Text:
    return "action_fetch_data"

def run(self, dispatcher: CollectingDispatcher,
        tracker: Tracker,
        domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
    output=FetchData(tracker.latest_message['entities'][0]['value'],
                     tracker.latest_message['entities'][1]['value'])
    dispatcher.utter_message(text="This is the data that you asked for, \n{}".format(",".join(output)))

    return []

In the above run method, you will see a function is called which will be created in the other script to read the data from the excel sheet,

def FetchData(column,occupation):
    if os.path.isfile("user_data.xlsx"):
        df=pd.read_excel("user_data.xlsx")
        data=df[column][df["occupation"]==occupation]
        return data.to_list()
    else:
        return ["There is no data. "]

Once, you have created the function for reading the data as per the user input. Also, add the stories in the stories.md file so as to follow the path,

## fetch info
* ask_info{"column":"email","occup":"trainer"}
  - action_fetch_data

Also add this action name to domain file to register it,

actions:
- action_fetch_data

For full clarification you can also check this session,

You can also download the full code here,

DOWNLOAD FULL CODE HERE

Once everything is done. Train the model and test it with the following commands running is separate terminal,

rasa train && rasa x
rasa run actions
docker run -p 8000:8000 rasa/duckling

This is how you bot will respond once everything is working fine,

Also, check these blogs if you want your chatbot to run on Telegram or facebook messenger.

Now, I hope everthing is understood and now you can apply this into your bussiness model to increase your reach. For any queries related to the topic feel free to leave a comment below. For more such useful contents check other blogs as well.

Stay connected and Happy learning. 🙂

21 thoughts on “BUILD AN END-USER CHATBOT

  1. Hi this is a wonderful article.
    I downloaded the code from GitHub and run it I am able to give details and data is storing as well. but when I am trying to retrieve the data from excel it’s showing me below error.

    Exception occurred while handling uri: ‘http://localhost:5055/webhook’
    Traceback (most recent call last):
    File “/home/rahul/venv/lib/python3.7/site-packages/sanic/app.py”, line 976, in handle_request
    response = await response
    File “/home/rahul/venv/lib/python3.7/site-packages/rasa_sdk/endpoint.py”, line 102, in webhook
    result = await executor.run(action_call)
    File “/home/rahul/venv/lib/python3.7/site-packages/rasa_sdk/executor.py”, line 387, in run
    events = action(dispatcher, tracker, domain)
    File “/home/rahul/Downloads/RasaChatbotExcelData-master/actions.py”, line 45, in run
    tracker.latest_message[‘entities’][1][‘value’])
    IndexError: list index out of range

    Your help is highly appreciated.

    Thank you

    Regards
    Rahul

      1. got it. I am actually looking for fetching values that are stored in my sql. Can you please do a tutorial where we can fetch details from my sql or sql server.

        Thank you

          1. Hi, This is really helpful thanks a lot. Can you also make a video where we can have interactions with external API services?

      2. Hello sir i am getting error while using duckling can you please help me. I think my rasa/duckling is working as it gives output as : Listening on http://0.0.0.0:8000

        Then I also made sure I did rasa run actions.

        And when I finally do rasa shell, it works fine and then if I give any input it shows :

        ERROR rasa.nlu.extractors.duckling_http_extractor – Failed to connect to duckling http server. Make sure the duckling server is running/healthy/not stale and the proper host and port are set in the configuration. More information on how to run the server can be found on github: https://github.com/facebook/duckling#quickstart Error: (‘Connection aborted.’, OSError(0, ‘Error’))

  2. No, for example within a weather bot (external API) where we can do CRUD operations via interactions. Can you make a video on these topics? it will be really helpful.

    Thank you

  3. Traceback (most recent call last):
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    domain.py”, line 546, in index_for_action
    return self.action_names.index(action_name)
    ValueError: None is not in list

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\runpy.py”, line 193, in
    _run_module_as_main
    “__main__”, mod_spec)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\runpy.py”, line 85, in _
    run_code
    exec(code, run_globals)
    File “C:\Users\Sathish\anaconda3\envs\pythonProject12\Scripts\rasa.exe\__main__.py
    “, line 7, in
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\__mai
    n__.py”, line 92, in main
    cmdline_arguments.func(cmdline_arguments)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\cli\t
    rain.py”, line 76, in train
    additional_arguments=extract_additional_arguments(args),
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
    .py”, line 50, in train
    additional_arguments=additional_arguments,
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\asyncio\base_events.py”,
    line 587, in run_until_complete
    return future.result()
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
    .py”, line 101, in train_async
    additional_arguments,
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
    .py”, line 188, in _train_async_internal
    additional_arguments=additional_arguments,
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
    .py”, line 223, in _do_training
    additional_arguments=additional_arguments,
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\train
    .py”, line 361, in _train_core_with_validated_data
    additional_arguments=additional_arguments,
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    train.py”, line 66, in train
    agent.train(training_data, **additional_arguments)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    agent.py”, line 742, in train
    self.policy_ensemble.train(training_trackers, self.domain, **kwargs)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    policies\ensemble.py”, line 124, in train
    policy.train(training_trackers, domain, **kwargs)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    policies\memoization.py”, line 163, in train
    self._add_states_to_lookup(trackers_as_states, trackers_as_actions, domain)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    policies\memoization.py”, line 110, in _add_states_to_lookup
    feature_item = domain.index_for_action(action)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    domain.py”, line 548, in index_for_action
    self._raise_action_not_found_exception(action_name)
    File “c:\users\sathish\anaconda3\envs\pythonproject12\lib\site-packages\rasa\core\
    domain.py”, line 553, in _raise_action_not_found_exception
    f”Cannot access action ‘{action_name}’, ”
    NameError: Cannot access action ‘None’, as that name is not a registered action for
    this domain. Available actions are:
    – action_listen
    – action_restart
    – action_session_start
    – action_default_fallback
    – action_deactivate_form
    – action_revert_fallback_events
    – action_default_ask_affirmation
    – action_default_ask_rephrase
    – action_back
    – utter_ask_email
    – utter_ask_name
    – utter_ask_number
    – utter_goodbye
    – utter_greet
    – Form_Info

  4. Thanks for the amazing videos
    I want to deploy the basic rasa chatbot on some website using flask

    Just the basic one which come inbuilt with rasa.
    Can you please guide me how can I do it

  5. Hello sir i am getting error while using duckling can you please help me. I think my rasa/duckling is working as it gives output as : Listening on http://0.0.0.0:8000 Then I also made sure I did rasa run actions. And when I finally do rasa shell, it works fine and then if I give any input it shows : ERROR rasa.nlu.extractors.duckling_http_extractor – Failed to connect to duckling http server. Make sure the duckling server is running/healthy/not stale and the proper host and port are set in the configuration. More information on how to run the server can be found on github: https://github.com/facebook/duckling#quickstart Error: (‘Connection aborted.’, OSError(0, ‘Error’))

Leave a Reply

Your email address will not be published. Required fields are marked *