Serial-MQTT Bridge (console)

This command-line Python script connects a microcontroller program to a remote MQTT server over the network. It communicates with the microcontroller using the serial port, relaying lines of text to and from the MQTT server. This can be used as a platform for remote collaboration between embedded devices.

This tool is much simpler and has fewer dependencies that the graphical Arduino-MQTT Bridge (PyQt5). But as a result, it is not interactive and the settings must be must be customized before using.

The script is provided all in one file and can be be directly downloaded from mqtt_serial_bridge.py. The following sections have both documentation and the full code.

Installation Requirements

The code requires a working installation of Python 3 with pySerial and paho-mqtt. For suggestions on setting up your system please see Python 3 Installation.

User Guide

The procedure for use generally follows this sequence:

  1. Install one of the remote connection examples on your CircuitPython board. These can serve as starting points for your own sketches.

  2. Open the mqtt_serial_bridge.py program in a Python editor.

  3. Locate the top section named “Configuration” and follow the prompts to customize the settings.

  4. Run the script using Python 3, typically as follows: python3 mqtt_serial_bridge.py

  5. To stop the script, type Control-C.

If testing locally, it is convenient also to run the MQTT Monitor (PyQt5) in order to simulate the collaborating system.

Typical Output

The script will print messages while running. A typical session involving bidirectional communication might look something like this:

MQTT connected with flags: {'session present': 0}, result code: 0
Entering Arduino event loop for /dev/cu.usbmodem333101.  Enter Control-C to quit.
Received from serial device: 0 0 0 0 0 0
Received from serial device: 0 0 0 0 0 0
message received: topic: partner payload: b'0 0 1 0 1 0'
Received from serial device: 0 0 0 0 0 0
Received from serial device: 0 0 1 0 0 0
message received: topic: partner payload: b'0 0 1 0 0 0'
Received from serial device: 0 0 1 0 0 0
Received from serial device: 0 0 0 0 0 0
Received from serial device: 0 0 0 0 0 0
message received: topic: partner payload: b'0 0 1 0 0 1'
Received from serial device: 0 0 0 0 0 0
^CKeyboard interrupt caught, closing down...

Full Code

  1#!/usr/bin/env python3
  2"""mqtt_serial_bridge.py
  3Command-line utility to connect a serial-port device with a remote MQTT server.
  4Please note: the Configuration settings must be customized before us.
  5"""
  6
  7################################################################
  8# Configuration
  9
 10# The following variables must be customized to set up the network and serial
 11# port settings.
 12
 13# IDeATe MQTT server name.
 14mqtt_hostname = "mqtt.ideate.cmu.edu"
 15
 16# IDeATe MQTT server port, specific to each course.
 17# Please see https://mqtt.ideate.cmu.edu for details.
 18mqtt_portnum  = 8889   # 16-376
 19
 20# Username and password, provided by instructor for each course.
 21mqtt_username = ''
 22mqtt_password = ''
 23
 24# MQTT publication topic.  This is usually the students Andrew ID.
 25mqtt_topic = ''
 26
 27# MQTT receive subscription.  This is usually the partner Andrew ID.
 28mqtt_subscription = 'unspecified'
 29
 30# Serial port device to bridge to the network (e.g. Arduino).
 31# On Windows, this will usually be similar to 'COM5'.
 32# On macOS, this will usually be similar to '/dev/cu.usbmodem333101'
 33serial_portname = ''
 34
 35################################################################
 36################################################################
 37# Written in 2018-2021 by Garth Zeglin <garthz@cmu.edu>
 38
 39# To the extent possible under law, the author has dedicated all copyright
 40# and related and neighboring rights to this software to the public domain
 41# worldwide. This software is distributed without any warranty.
 42
 43# You should have received a copy of the CC0 Public Domain Dedication along with this software.
 44# If not, see <http://creativecommons.org/publicdomain/zero/1.0/>.
 45
 46################################################################
 47# Import standard Python libraries.
 48import sys, time, signal
 49
 50# Import the MQTT client library.
 51# documentation: https://www.eclipse.org/paho/clients/python/docs/
 52import paho.mqtt.client as mqtt
 53
 54# Import the pySerial library.
 55# documentation: https://pyserial.readthedocs.io/en/latest/
 56import serial
 57
 58################################################################
 59# Global script variables.
 60
 61serial_port = None
 62client = None
 63
 64################################################################
 65if mqtt_username == '' or mqtt_password == '' or mqtt_topic == '' or serial_portname == '': 
 66    print("""\
 67This script must be customized before it can be used.  Please edit the file with
 68a Python or text editor and set the variables appropriately in the Configuration
 69section at the top of the file.
 70""")
 71    if serial_portname == '':
 72        import serial.tools.list_ports
 73        print("All available serial ports:")
 74        for p in serial.tools.list_ports.comports():
 75            print(" ", p.device)
 76    sys.exit(0)
 77
 78################################################################
 79# Attach a handler to the keyboard interrupt (control-C).
 80def _sigint_handler(signal, frame):
 81    print("Keyboard interrupt caught, closing down...")
 82    if serial_port is not None:
 83        serial_port.close()
 84
 85    if client is not None:
 86        client.loop_stop()
 87        
 88    sys.exit(0)
 89
 90signal.signal(signal.SIGINT, _sigint_handler)        
 91################################################################
 92# MQTT networking functions.
 93
 94#----------------------------------------------------------------
 95# The callback for when the broker responds to our connection request.
 96def on_connect(client, userdata, flags, rc):
 97    print(f"MQTT connected with flags: {flags}, result code: {rc}")
 98
 99    # Subscribing in on_connect() means that if we lose the connection and reconnect then subscriptions will be renewed.
100    # The hash mark is a multi-level wildcard, so this will subscribe to all subtopics of 16223
101    client.subscribe(mqtt_subscription)
102    return
103#----------------------------------------------------------------
104# The callback for when a message has been received on a topic to which this
105# client is subscribed.  The message variable is a MQTTMessage that describes
106# all of the message parameters.
107
108# Some useful MQTTMessage fields: topic, payload, qos, retain, mid, properties.
109#   The payload is a binary string (bytes).
110#   qos is an integer quality of service indicator (0,1, or 2)
111#   mid is an integer message ID.
112def on_message(client, userdata, msg):
113    print(f"message received: topic: {msg.topic} payload: {msg.payload}")
114
115    # If the serial port is ready, re-transmit received messages to the
116    # device. The msg.payload is a bytes object which can be directly sent to
117    # the serial port with an appended line ending.
118    if serial_port is not None and serial_port.is_open:
119        serial_port.write(msg.payload + b'\n')
120    return
121
122#----------------------------------------------------------------
123# Launch the MQTT network client
124client = mqtt.Client()
125client.enable_logger()
126client.on_connect = on_connect
127client.on_message = on_message
128client.tls_set()
129client.username_pw_set(mqtt_username, mqtt_password)
130
131# Start a background thread to connect to the MQTT network.
132client.connect_async(mqtt_hostname, mqtt_portnum)
133client.loop_start()
134
135################################################################
136# Connect to the serial device.
137serial_port = serial.Serial(serial_portname, baudrate=115200, timeout=2.0)
138
139# wait briefly for the Arduino to complete waking up
140time.sleep(0.2)
141
142print(f"Entering Arduino event loop for {serial_portname}.  Enter Control-C to quit.")
143
144while(True):
145    input = serial_port.readline().decode(encoding='ascii',errors='ignore').rstrip()
146    if len(input) == 0:
147        print("Serial device timed out, no data received.")
148    else:
149        print(f"Received from serial device: {input}")
150        if client.is_connected():
151            client.publish(topic=mqtt_topic, payload=input)