Voice platforms like Alexa and Google Assistant make it easy to provide a custom voice experience to your clients, even without going deeper in audio processing — everything is part of the platform. But what if you already invested quite some effort into building a chatbot on SAP Conversational AI ? You certainly don’t want to switch to a totally new platform now.

Image for post
Image for post
Voice Testing with Botium and SAP Conversational AI

Head on to my blog article in the SAP Community!


A quick summary of 7 important DO’s and DON’Ts when designing a chatbot testing strategy. We are continuously seeing teams ignoring those actually rather simple rules.

Image for post
Image for post

DO’s and DON’Ts

DO plan for iterations

In german we say rome was not built in a day — same applies for your chatbot training data. A robust chatbot is built by multiple iterations, training and testing cycles and by ongoing monitoring and performance tuning: CODE, TEST, DEPLOY, REPEAT

DON’T underestimate the need for constant performance measurement

Without measuring performance with real user conversations, you will never know if your chatbot is really working for your users.

✅DO apply the 80/20 rule for testing utterances

Most teams are tempted to use 100% of the available data for…


This guide suggests best practices, infrastructure and tools to ensure your voice app continues to deliver outstanding user experience.

Image for post
Image for post

Questions When Testing Voice Apps

Application of the suggested practices helps answering the questions:

  • Is my voice app following the designed conversation flow ? Is the conversation flow working as expected ?
  • How does my voice app work under real-life conditions ? Does is handle low audio quality ? Does it handle slow network connections ?
  • Is my voice app available 24x7, or are there any interruptions in service ?

The Art of Challenging Chatbots

The challenges when testing chatbots, escpecially voice-enabled ones, are different ones than when testing apps with…


Voice platforms like Alexa and Google Assistant make it easy to build your own voice experience, even without going deeper in audio processing — everything is part of the platform. But what if you want to rather go for a solution hosted by yourself, running an assistant on your own website, in your own infrastructure ?

The goal of this article is to show a way how you can build your own voice platform using Open Source tools Rasa and Botium Speech Processing.

Image for post
Image for post
An awesome isometric visualization

Rasa is a developer-friendly and extensible chatbot building tool for self-hosting. Botium Speech Processing is a unified…


Botium delivers again — in this case, automated end-2-end testing of WhatsApp chatbots on real or virtual devices. For the first time it is now possible to have a full enterprise-level test strategy for WhatsApp chatbots.

Image for post
Image for post

Automated vs Manual Testing

When it comes to testing WhatsApp chatbots up to now there have been mainly two approaches:

  • Testing manually on a smartphone
  • Testing backend functionality with API Testing

Both approaches are valid and no enterprise-level test strategy should miss any of them. But there are two obvious flaws:

  1. Manual testing is not scalable to the extent that is required to guarantee constant high quality for…


This article is pointing out security threats and attack vectors of typical chatbot architectures — based on OWASP Top 10 and adversarial attacks.

Image for post
Image for post

The well-known OWASP Top 10 is a list of top security threats for a web application. Most chatbots out there are available over a public web frontend, and as such all the OWASP security risks apply to those chatbot frontends as well. …


What are Embeddings? What is similarity, cohesion and separation?

UPDATE 2020/11/01: Botium’s free plan is live! With Botium Box Mini you will be able to:

  • use multiple chatbot technologies
  • set up test automation in a few minutes
  • enjoy a new improved user interface
  • get the benefits of a hosted, free service

Take it for a test drive

This article series provides an introduction to important quality metrics for your NLU engine and your chatbot training data. …


UPDATE 2020/11/01: Botium’s free plan is live! With Botium Box Mini you will be able to:

  • use multiple chatbot technologies
  • set up test automation in a few minutes
  • enjoy a new improved user interface
  • get the benefits of a hosted, free service

Take it for a test drive

All steps are demonstrated in the video tutorial.

Step 1: Connect Botium Box to your Rasa endpoint

Image for post
Image for post

Step 2: Import the Rasa training data nlu.md as Botium Test Set


UPDATE 2020/11/01: Botium’s free plan is live! With Botium Box Mini you will be able to:

  • use multiple chatbot technologies
  • set up test automation in a few minutes
  • enjoy a new improved user interface
  • get the benefits of a hosted, free service

Take it for a test drive

This tutorial provides a step-by-step introduction how to use Botium Box to quickly build a regression test suite for an Amazon Lex chatbot and increase the test coverage by adding paraphrases to the test data.

What we will do:

  1. Create an Amazon Lex chatbot and enable programmatic access
  2. Connect Botium Box to…


UPDATE 2020/11/01: Botium’s free plan is live! With Botium Box Mini you will be able to:

  • use multiple chatbot technologies
  • set up test automation in a few minutes
  • enjoy a new improved user interface
  • get the benefits of a hosted, free service

Take it for a test drive

According to the report The State of AI 2019: Divergence there is a lot of Bullshit Bingo around the thing called Artificial Intelligence (AI). From page 101 of the report:

“In approximately 60% of the cases (…) there was no evidence of AI material to a company’s value proposition.”

Image for post
Image for post
Is this the state of AI in 2019 ? Hopefully not.

This means on…

Florian Treml

Co-Founder and CTO Botium🤓 — Guitarist 🎸 — 3xFather 🐣

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store