Botium in a Nutshell, Part 5: About Botium Box

Botium Box Taxonomy

As a starter a short overview of the terms used in this article an in the Botium Box:

  • Test Cases are written as Convos and Utterances — see here
  • A Test Set is a collection of Test Cases
  • A chatbot in Botium Box is a set of capabilities to tell Botium how to connect to a single instance of a chatbot
  • A test project is a combination of test sets and a chatbot — the specification of a test environment
  • A test session is the process of sending test sets to a chatbot and asserting the correctness of the results
  • A device set is a collection of device or browser specifications to be used for user interface testing
  • An agent is a piece of software for test execution. The more agents are connected to the Botium Box, the more tests can run in parallel.

The Dashboard

The Dashboard is the entry point. You can see an overview how the quality of your chatbot evolves over time, how the chatbot performs in your environments and what test sets perform well.

Botium Box Dashboard

Connecting your chatbot

Botium supports a large variety of chatbot technologies. For each technology, the configuration form contains inline help how to retrieve the required connection information, tokens, secrets etc.

Chatbot capability configuration screen

Live Chat

For checking your configuration, you can always start a live chat with your chatbot.

Starting the live chat
Live Chat screen
Save conversation as Test Case

The Test Set Library and Humanification

Test case files are managed in test set libraries. Within a single test set, you can combine test cases from various sources:

  • Write Convos and Utterances in Botium Box
  • Upload Excel files
  • Register a shared folder holding the test cases
  • Register a Git repository holding the test cases
Test Set view
  • space and newline handling
  • sausage fingers
  • common typographic errors for qwertz/qwerty keyboards
  • emojis
  • mishits
  • case sensitivity
  • and many more …
Humanification algorithms

Starting a Test Session

On the Test Results screen, a test session can be started. Depending on the number of test agents, the performance of the chatbot and the number of testcases this will take some time.

Starting a test session

Exploring the Test Results

The test report shows on a very detailed level every aspect of the test execution:

  • Successful and failed test cases
  • Transcripts for all executed test cases
  • Detailed log files for all test execution jobs
  • Performance measurement
Test Results
Test Result Downloads

Test Projects

Use the Quickstart Wizard to combine a chatbot and test sets into a test project.

Quickstart Wizard
Start test session
  • Botium CLI
  • A simple HTTP endpoint for triggering a test project
  • Copy&Paste scripts with cURL or wget
Build pipeline integration

Device Labs

Botium Box also supports testing on a user interface level — check that your chatbot is responsive on real devices by combining Botium with a device cloud and browser lab provider like Saucelabs.

Frequently Asked Questions

Please have a look at the Botium website.

Looking for contributors

Please take part in the Botium community to bring chatbots forward! By contributing you help in increasing the quality of chatbots worldwide, leading to increasing end-user acceptance, which again will bring your own chatbot forward! Start here:



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Florian Treml

Florian Treml

Co-Founder and CTO Botium🤓 — Guitarist 🎸 — 3xFather 🐣