UPDATE 2020/11/01: Botium’s free plan is live! With Botium Box Mini you will be able to:
- use multiple chatbot technologies
- set up test automation in a few minutes
- enjoy a new improved user interface
- get the benefits of a hosted, free service
This article is pointing out security threats and attack vectors of typical chatbot architectures — based on OWASP Top 10 and adversarial attacks.
The well-known OWASP Top 10 is a list of top security threats for a web application. Most chatbots out there are available over a public web frontend, and as such all the OWASP security risks apply to those chatbot frontends as well. Out of these risks there are two especially important to defend against, as in contrary to the other risks, those two are nearly always a serious threat when talking about chatbots — XSS and SQL Injection.
Recently another kind of security threat came up, specifically targeting NLP models — so-called “adversarial attacks”.
Cross-Site Scripting - XSS
A typical implementation of a chatbot frontend:
- There is a chat window with an input box
- Everything the user enters in the input box is mirrored in the chat window
- Chatbot response is shown in the chat window
Possible Chatbot Attack Vector
For exploiting an XSS vulnerability the attacker has to trick the victim to send malicious input text.
- Attacker tricks the victim to click a hyperlink pointing to the chatbot frontend including some malicious code in the hyperlink
- The malicious code is injected into the website
- It reads the victims cookies and sends it to the attacker without the victim even noticing
- The attacker can use those cookies to get access to the victim’s account on the company website
SQL Injection — SQLI
A typical implementation of a task-oriented chatbot backend:
- User tells the chatbot some information item
- The chatbot backend queries a data source for this information item
- Based on the result a natural language response is generated and presented to the user
With SQL Injection, the attacker may trick the chatbot backend to consider malicious content as part of the information item:
my order number is "1234; DELETE FROM ORDERS"
Developers typically trust their tokenizers and entity extractors to defend against injection attack.
Possible Chatbot Attack Vector
When the attacker has personal access to the chatbot frontend, an SQL injection is exploitable directly by the attacker (see example above), doing all kind of SQL (or no-SQL) queries .
This is a new type of attack specifically targeting at classifiers — the NLP model backing a chatbot is basically a text classifier.
An adversarial attack tries to identify blind spots in the classifier by applying tiny, in worst case invisible changes (noise) to the classifier input data. A famous example is to trick an image classifier to a wrong classification by adding some tiny noise not visible for the human eye.
A more dangerous real-life attack is to trick an autonomous car to ignore a stop sign by adding some stickers to it.
The same concept can by applied to voice apps — some background noise not noticed by human listeners could trigger IoT devices in the same room to unlock the front door or place online shop orders.
When talking about text-based chatbots, the only difference is that it is not possible to totally hide added noise from the human eye, as noise in this case means changing single characters or whole words.
Possible Chatbot Attack Vector
For voice-based chatbots one possible risk is to hand over control to the attacker based on manipulated audio streams, exploiting weaknesses in the speech recognition and classification engine.
- The attacker tricks the victim to play a manipulated audio file from a malicious web site
- In the background, the voice device is activated and commands embedded into the audio file are executed
To be honest, it is hard to imagine a real-life security threat for text-based chatbots.
- User Experience is an important success factor for a chatbot. An NLP model not robust enough to handle typical human typing habits with typographic errors, character swapping, emojis provides a bad user experience, even without any malicious attacker involved.
- It could be possible to trick a banking chatbot to doing transactions with some hidden commands and at the same time deny that the transaction was wanted, based on the chatbot logs … (I know, not that plausible …)
Security and Penetration Testing with Botium Box
Botium Box includes several tools for improving the robustness of your chatbot and your NLP model against the attacks above.
Penetration Testing with OWASP ZAP Zed Attack Proxy
Botium Box provides a unique way of running continuous security tests based on the OWASP ZAP Zed Attack Proxy — read more in the Botium Wiki. It helps to identify security vulnerabilities in the infrastructure, such as SSL issues and outdated 3rd-party-components.
E2E Test Sets for SQL Injection and XSS
Botium Box includes test sets for running End-2-End-Security-Tests on device cloud and browser farms, based on OWASP recommendations:
- Over 70 different XSS scenarios
- More than 10 different SQL Injection scenarious
- Exceptional cases like character encoding, emoji flooding and more
The Botium Humanification Layer checks your NLP model for robustness against adversarial attacks and common human typing behaviour:
- simulating typing speed
- common typographic errors based on keyboard layout
- punctuation marks
- and more …
Read more in the Botium Wiki.
With the paraphraser it is possible increase test coverage with a single mouse click — read on here:
Tutorial: Using Paraphrasing to Increase Conversational AI Test Coverage
This tutorial provides a step-by-step introduction how to use Botium Box to quickly build a regression test suite for…
With Botium Box Load Testing and Stress Testing you can simulate user load on your chatbot and see how it behaves under production load — read on in the Botium Wiki.
Give Botium Box a test drive today — start with the free trial, we are happy to hear from you if you find it useful!
Looking for contributors
Please take part in the Botium community to bring chatbots forward! By contributing you help in increasing the quality of chatbots worldwide, leading to increasing end-user acceptance, which again will bring your own chatbot forward! Start here