MQTT stands for Message Queuing Telemetry Transport. It is a messaging protocol for IoT devices with resource constraints. It is specialized for low bandwidth and high latency environments, making it an ideal protocol for machine-to-machine (M2M) communication.

MQTT works according to the publisher/ subscriber principle with a central broker. This principle means that the sender and receiver have no direct connection. The senders publish the data on a topic, and all recipients who have subscribed to this topic receive the data.

In this post, we will introduce the basic functionalities of MQTT. We also show how to set up a secure MQTT broker with Docker. In a second article, we will create an MQTT Publisher and Subscriber in Python using this MQTT Broker. So be curious about the second article. But now we start with the first article. 😉

The steps are the following:

1. MQTT basics

2. Technical requirements

3. Set up an MQTT Broker with Docker

4. MQTT Explorer

5. Conclusion

6. Useful links

---

🎓 Our Online Courses and recommendations

---

🤔 MQTT basics

The MQTT protocol connects IoT devices. We can use it for various purposes, such as Smart Homes or the Industrial Internet of Things (IIoT). There are sensors (publishers), actuators (subscribers) and a broker (topics). An example is below.

Smart Home (Image by authors)

On the left side, we see the sensors: door contact, temperature and light. These sensors publish their values on an MQTT broker. An MQTT broker is a server that manages the topics. The topics are window status, room temperature and light intensity. On the right side, we have the actuators: heating control and roller shutter control. The actuators react to sensor values.

Example 1: Heating only starts when the temperature is too low and there is no open window.

Example 2: The shutters only go down when the light intensity is low and there is no open window.

With MQTT, you can automate your home or industrial processes with less effort. Furthermore, MQTT offers different quality of service levels (QoS).

QoS levels:

• once at most (0): Message sent once at most.

• at least once (1): Message transfer at least once.

• exactly once (2): Message transmit exactly once.

A broker usually supports all QoS levels. In the publisher and subscriber, you can set the QoS levels. That was enough theory! Let’s continue with the practical part. 😀

✅ Technical requirements

You will need the following prerequisites:

• The latest version of Docker must be installed on your machine. If you do not have it installed yet, please follow the instructions.

• The latest version of Docker Compose must be installed on your machine. Please follow the instructions.

• Access to a bash (macOS, Linux or Windows).

🖥 Set up an MQTT Broker with Docker

First, you should check that you have Docker installed correctly. Enter the following command in a terminal of your choice:

$ docker --version
# Example output: $ Docker version 20.10.22

If the installation is correct, you can see the Docker version (Maybe you already have a newer Docker version). You can check the same for the Docker Compose installation.

$ docker-compose --version
# Example output: $ Docker Compose version v2.15.1

Yeah. 😀 Everything is okay. The Docker installation was successful.

Next, we set up a secure MQTT broker with Docker. Create a folder to store all the files from this tutorial. For example, call the folder broker. In this folder, you create a file with the name docker-compose.yml. Please add the following content to the file.

version: '3.8'

services:
  mosquitto:
    container_name: mosquitto-mqtt
    image: eclipse-mosquitto:latest
    volumes:
      - ./config:/mosquitto/config/
      - mosquitto_data:/mosquitto/data
      - mosquitto_log:/mosquitto/log
    ports:
      - '1883:1883'
      - '8883:8883'
    environment:
      TZ: 'Europe/Berlin'
    networks:
      - mqtt
    restart: always

networks:
  mqtt:

volumes:
  mosquitto_data:
  mosquitto_log:

Let’s take a look at the service in detail. First, we define the Docker Compose file format version, here 3.8. Then we define the service with the name mosquitto. After that, we set the name of the Docker Container, here mosquitto-mqtt. We use the Docker Image eclipse-mosquitto:latest from DockerHub.

Next, we come to the definition of the volumes. Please create a folder called config and a folder called keys in the directory /broker/. In the following, we discuss which files you need in the two folder keys und config.

Folder keys:

This folder contains the certificates and a script for generating the certificates. Let’s have a look at the script for generating the certificates (This is an example script.).

#!/bin/bash

# Relative Distinguished Names (RDNs).
# - CN: Common Name
# - OU: Organizational Unit
# - O: Organization
# - L: Locality
# - S: State Or Province Name
# - C: Country Name

# check which operating system (only macOS or linux)
if [[ "$OSTYPE" =~ ^darwin ]]; then
    COMMON_NAME=$(scutil --get ComputerName)
fi

if [[ "$OSTYPE" =~ ^linux ]]; then
    COMMON_NAME=$(hostname)
fi

echo ${COMMON_NAME}

SUBJECT_CA="/C=DE/ST=Berlin/L=Berlin/O=my-organisation/OU=CA/CN=$COMMON_NAME"
SUBJECT_SERVER="/C=DE/ST=Berlin/L=Berlin/O=my-organisation/OU=Server/CN=$COMMON_NAME"
SUBJECT_CLIENT="/C=DE/ST=Berlin/L=Berlin/O=my-organisation/OU=Client/CN=$COMMON_NAME"

function generate_CA ()
{
  echo "$SUBJECT_CA"
  openssl req -x509 -nodes -sha256 -newkey rsa:2048 -subj "$SUBJECT_CA" -days 365 -keyout ca.key -out ca.crt
}

function generate_server ()
{
  echo "$SUBJECT_SERVER"
  openssl req -nodes -sha256 -new -subj "$SUBJECT_SERVER" -keyout server.key -out server.csr
  openssl x509 -req -sha256 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365
}

function generate_client ()
{
  echo "$SUBJECT_CLIENT"
  openssl req -new -nodes -sha256 -subj "$SUBJECT_CLIENT" -out client.csr -keyout client.key
  openssl x509 -req -sha256 -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 365
}

function copy_keys_to_certs ()
{
  cp ca.crt ../config/certs/
  cp server.crt ../config/certs/
  cp server.key ../config/certs/
}

generate_CA
generate_server
generate_client
copy_keys_to_certs

In the script, we generate the certificates with OpenSSL. OpenSSL is an open-source implementation of the (SSL and) TLS protocol. First, we read out the computer name for Linux or Mac. We use this name later for the common name (CN). The CN is the fully qualified name for the system. For static DNS, use the hostname or IP address. Then we define the subject for the Certificate Authority (CA), the client and the server certificate. You can replace the entries in the subject individually. If you would like to take a closer look at the topic of encryption, you can read the following OpenSSL tutorial. The tutorial shows very clearly how encryption and signatures work.

In the next step, we define four functions. The explanation is below:

• generate_CA(): In this function, we generate the files for the CA ( ca.crt (public certificate) and ca.key (private key)). The CA certificate is valid for 365 days. You are welcome to choose a different number.

• generate_server(): This function creates a certificate request (CSR)(server.csr) containing the public key and some other information (name, address, organization, etc.). The CSR is then signed by a trusted third party (CA) using the CA’s private key ca.key and converted into a certificate (server.crt). -CAcreateserial creates a file ca.srl containing a counter of how many certificates have been signed by this CA. The server certificate is valid for 365 days. We get the two files: server.crt and server.key.

• generate_client(): We do the same for the client certificate as for the server certificate. The only difference is the name of the certificate. We get the two files: client.crt and client.key.

• copy_keys_to_certs(): Then we copy the relevant certificates for the MQTT broker into the config/certsfolder so that the broker can find the certificates.

Create a script with the name make_keys.sh in the folder /keys. Paste the code discussed above into the script and save it. Then execute the following command in the folder /keys in your terminal:

$ bash make_keys.sh

The script generates the certificates for the CA, the server and the client. In addition, we copy all relevant certificates for the server to the config folder. Next, we look at which files still need to be in the config folder.

Folderconfig:

This folder contains the MQTT broker configuration stuff. First, create a file called mosquitto.conf in the /configfolder. For more information on the mosquitto.conf file, see the mosquitto.org website. Add the following content to the file mosquitto.conf.

listener 1883
listener 8883

persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log

allow_anonymous false
password_file /mosquitto/config/mosquitto.passwd

cafile /mosquitto/config/certs/ca.crt
certfile /mosquitto/config/certs/server.crt
keyfile /mosquitto/config/certs/server.key

require_certificate true   # true: client needs certificates, false: client needs no certificates
use_identity_as_username false   # false: client authenticates as normal (user, password), for more see https://mosquitto.org/man/mosquitto-conf-5.html

First, we define that our broker will access ports 1883 (unencrypted) and 8883 (encrypted). The broker can operate in an unencrypted and an encrypted mode. We also set the persistence variable to true. If we set the variable to true, connection, subscription and message data are written to disk in mosquitto.db at the location specified by the Docker Compose file (volume mosquitto_data). We store the logs of the broker in the volume mosquitto_log. The variable allow_anonymous describes whether we allow anonymous login. We set the variable to false because we do not allow anonymous login to our MQTT broker. Next, we set the path to our password file. Please create a file with the name mosquitto.passwd in the /config folder. Open the file and add the following content.

mosquitto:$6$oF7jfj7Ex178nHBT$uGjmdm2GTQ9yKHPTSGJZnuJVUOmOTqlCD9R1hLWGfITv5Z/YdZDWbzZea5EECadwPlRuxO56wjeQbFj5Kp/UEg==

The file mosquitto.passwd sets the username to “mosquitto” and the password to “mosquitto”. You see the encrypted password.

Next, we add the path to the required certificates (ca.crt, server.crt and server.key). We set the variable require_certificate to true so that clients have to use certificates. We also activate the user/ password authentication.

In the next step, we specify the ports in the Docker Compose file. We also set the time zone. In our case, for example, Berlin. Furthermore, define a Docker network for the broker and specify that the container should always restart.

Your final file structure should look like this:

MQTT broker file structure (Screenshot by authors)

Now, you can start the MQTT broker with the following command in a terminal of your choice:

$ docker compose up -d

The flag -d means that the container is running as a daemon. In this mode, the terminal does not output any logs. Now the MQTT broker is started, and you can use it.

---

📖 Explore our premium blog articles

---

💻 MQTT Explorer

In this tutorial, we use the tool MQTT Explorer to test the functionality of the MQTT broker (You can also use another application of your choice.). You can download the application from the MQTT Explorer website.

First, we establish an unencrypted connection via port 1883. The following screenshot shows the login data.

Unencrypted connection via port 1883 (Screenshot by authors)

The following login data are default:

• Username: mosquitto

• Password: mosquitto

We strongly recommend changing the password!

You can change the password and the username with the following command (Please note that you must be in the folder with the Docker Compose file.):

$ docker-compose exec mosquitto mosquitto_passwd -c /mosquitto/config/mosquitto.passwd <username>

Please replace the variable <username> with a new username. Then enter the password twice. After that, you have to restart the MQTT broker. Enter the following commands in the terminal:

Stopping the MQTT broker:

$ docker-compose down

Starting the MQTT broker:

$ docker-compose up -d

Now, you can click Connect to access the broker.

Next, we connect to the encrypted broker. We enter the login data again, and the options “Validate certificate” and “Encryption (tls)” must be activated. See the following screenshot.

Encrypted connection via port 8883 (Screenshot by authors)

In addition, we have to set the certificates in the MQTT Explorer. Use the certificates from the folder /keys. The following screenshot shows the settings in the MQTT Explorer.

Certificates for the encrypted connection (Screenshot by authors)

If you set everything correctly, you can use the encrypted MQTT broker. Congratulations. Have fun with it. 🥳

🎬 Conclusion

In this article, you have learned how MQTT works and how you can use it. We also saw how to set up a secure MQTT broker with Docker. In this context, we created TLS certificates, which we finally used in the MQTT Broker and the client application (MQTT Explorer).

In the next article, we will create a publisher and a subscriber in Python. We show you how to include the certificates in your code and how the implementation works. As a result, we can send small data packets via the broker. Stay tuned!

---

👉🏽 Join our free weekly Magic AI newsletter for the latest AI updates!

👉🏽 Elevate your Python and ML skills with our top course recommendations!

Did you enjoy our content and find it helpful? If so, be sure to check out our premium offer! Don't forget to follow us on X. 🙏🏽🙏🏽

Thanks so much for reading. Have a great day!

---

🔍 Useful literature

Books

• Practical Python Programming for IoT: Build advanced IoT projects using a Raspberry Pi 4, MQTT, RESTful APIs, WebSockets, and Python 3

• Hands-On MQTT Programming with Python: Work with the lightweight IoT protocol in Python

Links

• Creating a Self-Signed Certificate With OpenSSL (german)

• Mosquitto Configuration

• Website MQTT Explorer

The authors are NOT LIABLE for any damages arising from the software. Use of the software is at your own risk.