Serial-MQTT Bridge (console)

This command-line Python script connects a microcontroller program to a remote MQTT server over the network. It communicates with the microcontroller using the serial port, relaying lines of text to and from the MQTT server. This can be used as a platform for remote collaboration between embedded devices.

This tool is much simpler and has fewer dependencies that the graphical Arduino-MQTT Bridge (PyQt5). But as a result, it is not interactive and the settings must be must be customized before using.

The script is provided all in one file and can be be directly downloaded from mqtt_serial_bridge.py. The following sections have both documentation and the full code.

Installation Requirements

The code requires a working installation of Python 3 with pySerial and paho-mqtt. For suggestions on setting up your system please see Python 3 Installation.

User Guide

The procedure for use generally follows this sequence:

  1. Install one of the remote connection examples on your CircuitPython board. These can serve as starting points for your own sketches.

  2. Open the mqtt_serial_bridge.py program in a Python editor.

  3. Locate the top section named “Configuration” and follow the prompts to customize the settings.

  4. Run the script using Python 3, typically as follows: python3 mqtt_serial_bridge.py

  5. To stop the script, type Control-C.

If testing locally, it is convenient also to run the MQTT Monitor (PyQt5) in order to simulate the collaborating system.

Typical Output

The script will print messages while running. A typical session involving bidirectional communication might look something like this:

MQTT connected with flags: {'session present': 0}, result code: 0
Entering Arduino event loop for /dev/cu.usbmodem333101.  Enter Control-C to quit.
Received from serial device: 0 0 0 0 0 0
Received from serial device: 0 0 0 0 0 0
message received: topic: partner payload: b'0 0 1 0 1 0'
Received from serial device: 0 0 0 0 0 0
Received from serial device: 0 0 1 0 0 0
message received: topic: partner payload: b'0 0 1 0 0 0'
Received from serial device: 0 0 1 0 0 0
Received from serial device: 0 0 0 0 0 0
Received from serial device: 0 0 0 0 0 0
message received: topic: partner payload: b'0 0 1 0 0 1'
Received from serial device: 0 0 0 0 0 0
^CKeyboard interrupt caught, closing down...

Full Code

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
#!/usr/bin/env python3
"""mqtt_serial_bridge.py
Command-line utility to connect a serial-port device with a remote MQTT server.
Please note: the Configuration settings must be customized before us.
"""

################################################################
# Configuration

# The following variables must be customized to set up the network and serial
# port settings.

# IDeATe MQTT server name.
mqtt_hostname = "mqtt.ideate.cmu.edu"

# IDeATe MQTT server port, specific to each course.
# Please see https://mqtt.ideate.cmu.edu for details.
mqtt_portnum  = 8889   # 16-376

# Username and password, provided by instructor for each course.
mqtt_username = ''
mqtt_password = ''

# MQTT publication topic.  This is usually the students Andrew ID.
mqtt_topic = ''

# MQTT receive subscription.  This is usually the partner Andrew ID.
mqtt_subscription = 'unspecified'

# Serial port device to bridge to the network (e.g. Arduino).
# On Windows, this will usually be similar to 'COM5'.
# On macOS, this will usually be similar to '/dev/cu.usbmodem333101'
serial_portname = ''

################################################################
################################################################
# Written in 2018-2021 by Garth Zeglin <garthz@cmu.edu>

# To the extent possible under law, the author has dedicated all copyright
# and related and neighboring rights to this software to the public domain
# worldwide. This software is distributed without any warranty.

# You should have received a copy of the CC0 Public Domain Dedication along with this software.
# If not, see <http://creativecommons.org/publicdomain/zero/1.0/>.

################################################################
# Import standard Python libraries.
import sys, time, signal

# Import the MQTT client library.
# documentation: https://www.eclipse.org/paho/clients/python/docs/
import paho.mqtt.client as mqtt

# Import the pySerial library.
# documentation: https://pyserial.readthedocs.io/en/latest/
import serial

################################################################
# Global script variables.

serial_port = None
client = None

################################################################
if mqtt_username == '' or mqtt_password == '' or mqtt_topic == '' or serial_portname == '': 
    print("""\
This script must be customized before it can be used.  Please edit the file with
a Python or text editor and set the variables appropriately in the Configuration
section at the top of the file.
""")
    if serial_portname == '':
        import serial.tools.list_ports
        print("All available serial ports:")
        for p in serial.tools.list_ports.comports():
            print(" ", p.device)
    sys.exit(0)

################################################################
# Attach a handler to the keyboard interrupt (control-C).
def _sigint_handler(signal, frame):
    print("Keyboard interrupt caught, closing down...")
    if serial_port is not None:
        serial_port.close()

    if client is not None:
        client.loop_stop()
        
    sys.exit(0)

signal.signal(signal.SIGINT, _sigint_handler)        
################################################################
# MQTT networking functions.

#----------------------------------------------------------------
# The callback for when the broker responds to our connection request.
def on_connect(client, userdata, flags, rc):
    print(f"MQTT connected with flags: {flags}, result code: {rc}")

    # Subscribing in on_connect() means that if we lose the connection and reconnect then subscriptions will be renewed.
    # The hash mark is a multi-level wildcard, so this will subscribe to all subtopics of 16223
    client.subscribe(mqtt_subscription)
    return
#----------------------------------------------------------------
# The callback for when a message has been received on a topic to which this
# client is subscribed.  The message variable is a MQTTMessage that describes
# all of the message parameters.

# Some useful MQTTMessage fields: topic, payload, qos, retain, mid, properties.
#   The payload is a binary string (bytes).
#   qos is an integer quality of service indicator (0,1, or 2)
#   mid is an integer message ID.
def on_message(client, userdata, msg):
    print(f"message received: topic: {msg.topic} payload: {msg.payload}")

    # If the serial port is ready, re-transmit received messages to the
    # device. The msg.payload is a bytes object which can be directly sent to
    # the serial port with an appended line ending.
    if serial_port is not None and serial_port.is_open:
        serial_port.write(msg.payload + b'\n')
    return

#----------------------------------------------------------------
# Launch the MQTT network client
client = mqtt.Client()
client.enable_logger()
client.on_connect = on_connect
client.on_message = on_message
client.tls_set()
client.username_pw_set(mqtt_username, mqtt_password)

# Start a background thread to connect to the MQTT network.
client.connect_async(mqtt_hostname, mqtt_portnum)
client.loop_start()

################################################################
# Connect to the serial device.
serial_port = serial.Serial(serial_portname, baudrate=115200, timeout=2.0)

# wait briefly for the Arduino to complete waking up
time.sleep(0.2)

print(f"Entering Arduino event loop for {serial_portname}.  Enter Control-C to quit.")

while(True):
    input = serial_port.readline().decode(encoding='ascii',errors='ignore').rstrip()
    if len(input) == 0:
        print("Serial device timed out, no data received.")
    else:
        print(f"Received from serial device: {input}")
        if client.is_connected():
            client.publish(topic=mqtt_topic, payload=input)