Botium in a Nutshell, Part 9: Botium Test Levels or Selecting a Botium Connector

This is part 9 of the Botium in a Nutshell series of articles.

Botium supports pretty much every important chatbot technology out there. There are over 20 Botium Connectors available, the number is growing by community and partner contributions.

This article gives an overview what kind of test levels are supported by Botium, and the strengths and weaknesses of each.

Common Chatbot Architecture

  • There is a user frontend hosted as chat widget on a website
  • The frontend connects to a backend service (often called Orchestration Service) with web-protocols — HTTP(S), JSON, Websockets
  • Somewhere behind, there is an NLU/NLP engine to convert user input to structured data with intents and entities
  • An additional component is handling the dialogue sessions.
  • An finally, there are some kind of business services, often backed by business databases

Note that in most cases the NLU/NLP engine and the Dialogue engine are combined, for example with IBM Watson Assistant, or Dialogflow. For Microsoft LUIS and Azure Bot Service, those components are spearated though.

The architecture diagrom above shows 6 integration points for Botium test cases.

1: User Interface Level / E2E Testing

Scenarios

  • Smoke Tests before going live

Pros

  • Works for all kind of chatbots, independent of the backend technology (in theory)

Cons

  • Usually, Webdriver scripting is required (in Javascript)
  • Very slow compared to the other testing levels

Part 6 of this series deals with E2E-Testing with Botium

2: API Level / Orchestration Service Testing

Scenarios

  • Integration Testing

Pros

  • Testing near end user experience

Cons

3+4: Backend Level / Dialogue and NLU/NLP Engine Testing

Botium provides connectors for all important dialogue engines out there, as well as for well-known NLP/NLU engines(see Botium Connectors list), and includes a module for providing NLP/NLU analytics.

Scenarios

  • NLP/NLU analytics

Pros

  • Allows very deep analysis of NLP data such as intents and entities

Cons

5: Business Logic Level / Business Service Testing

  • By using an eCommerce chatbot, Botium places a test order with the help of an order service in the backend
  • The order service persists the order in the business database
  • Botium asks the order service for details about the order to verify that the order has been persisted

6: Business Data Level / Business Database Testing

Looking for contributors

Co-Founder and CTO Botium🤓 — Guitarist 🎸 — 3xFather 🐣

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store