# BLE gateway
The manufacturer agnostic BLE gateway acts as a powerful BLE scanner and decoder of Bluetooth devices (opens new window), allowing you to visualize and analyze information from a wide range of sensors. It can also act as a presence detection gateway by reading the nearby BLE tags or tracker.
Data are transmitted to an MQTT broker, where it can be used to trigger events and rules, as well as displayed, stored and processed in your favorite controller (Home Assistant, OpenHAB, Jeedom, Domoticz, ioBroker or any MQTT compatible software).
With the ability to monitor and analyze data such as temperature, humidity, moisture, luminance, weight, pressure, fine particles, and more, the BLE gateway provides a flexible and customizable solution for integrating BLE technology into your control and monitoring systems.
# The first plug-and-play OpenMQTTGateway BLE gateway and Smart Plug!
Theengs plug (opens new window) is available flashed with OpenMQTTGateway, and brings the functions below:
- BLE to MQTT gateway, tens of Bluetooth devices (opens new window) supported thanks to Theengs Decoder library. The plug uses an ESP32 acting as a BLE to Wifi gateway to scan, decode and forward the data of the nearby sensors,
- Smart plug that can be controlled remotely,
- Energy consumption monitoring,
- Presence detection (beta).
Support the project by purchasing the Theengs plug (opens new window) The plug is available in North America only, other regions are planned.
# Disable or enable the BLE gateway (default: true, available with HA discovery)
If you want to deactivate the BLE gateway:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"enabled":false}'
If you want to activate the BLE gateway:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"enabled":true}'
# Receiving signals from compatible BLE sensors (opens new window) to publish it to an MQTT broker.
To receive data from BLE sensors you can use an ESP32-based device with a programming USB port or use a Serial adapter.
OpenMQTTGateway is also available preloaded and configured with the Theengs plug (opens new window), a smart plug that acts as a BLE gateway and energy monitoring device.
Follow the Upload and Configuration steps.
Download an MQTT client like MQTT explorer.
You should see data coming in your broker.
Once the data has been transmitted to the MQTT broker, it can be easily integrated with your preferred controller. For example, the data can be automatically discovered and made available within popular controllers, example below with Home Assistant.
Examples of compatible sensors among [our list](https://decoder.theengs.io/devices/devices_by_brand.html: Mi Flora, Mi jia, LYWDS02, LYWSD03MMC, ClearGrass, Mi scale, iBBQ, TPMS
# Receiving signals from BLE tracker devices for Presence detection
The gateway can detect BLE trackers from Tile, Nut, TagIt and iTag, as well as other devices with additional properties decoding like Mi Band, Amazfit, RuuviTag and others indicated as Presence Trackers in the compatible BLE devices list (opens new window), and automatically creates a device tracker entity following the Home Assistant discovery convention (if auto discovery is activated).
To do this activate the "BT: Publish HASS presence" switch in your controller or send the following MQTT command to your broker:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"hasspresence":true}'
The entity created can be attached to a person to leverage presence detection. The away or not home state is triggered if the BLE tracker is not detected during the timer defined by presenceawaytimer.
If you have multiple gateways, your BLE trackers may not be detected temporary by one gateway but still by the others. In this case you will see the tracker appears offline briefly and online again once it is detected by the others gateways.
By default presenceawaytimer is set to 120s, you can change it from the slider in your controller or with the following command (ms)
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"presenceawaytimer":66000}'
Generally BLE devices will not broadcast if they are paired so you may need to ensure your beacons is unpaired/disconnected before it will be seen by the gateway.
Consider the distance estimation as a beta feature.
Note that you can find apps to simulate beacons and do some tests like Beacon simulator (opens new window)
iOS version >=10 devices advertise without an extra app MAC address, nevertheless this address changes randomly (opens new window) and cannot be used for presence detection. You must install an app to advertise a fixed MAC address.
::: info
The presenceawaytimer is also used to reset the state of the PIR/motion sensors to off when using HA MQTT discovery convention. If the Sensor does not detect a motion, its state will be automatically set to off after the presenceawaytimer.
:::
# Receiving signals from BLE devices with accelerometers for movement detection
The gateway is designed to detect BLE trackers from BlueCharm and automatically create a binary sensor entity in accordance with the Home Assistant discovery convention, provided that auto discovery is enabled.
The binary sensor entity's state will be set to on or off based on whether the BLE beacon's x-axis acceleration value is detected during the time period defined by the movingtimer parameter.
By default, the movingtimer parameter is set to 60 seconds (60,000 ms). To modify this value, use the following command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"movingtimer":66000}'
To ensure proper functionality, configure the beacon using the KBeacon or KBeaconPro app (depending on your specific sensor) with the following settings:
- In the General tab, set the Trigger Command to "motion".
- In the Slot tab, enable advertisement with the Beacon Type set to "KSensor", the Trigger Only Advertisement option set to "YES", and the Sensor Axis set to "ON".
# Setting a white or black list
A black list is a list of MAC addresses that will never be published by OMG
to set black list
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"black-list":["01:23:14:55:16:15","4C:65:77:88:9C:79","4C:65:A6:66:3C:79"]}'
A white list is a list of MAC addresses permitted to be published by OMG
to set white list
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"white-list":["01:23:14:55:16:15","4C:65:77:88:9C:79","4C:65:A6:66:3C:79"]}'
Note: if you want to filter (white or black list) on BLE sensors that are auto discovered, you need to wait for the discovery before applying the white or black list, or temporarily disable it:
to temporarily disable white/black list
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"ignoreWBlist":true}'
to enable white/black list back
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"ignoreWBlist":false}'
TIP
So as to keep your white/black list persistent you can publish it with the retain option of MQTT (-r with mosquitto_pub or retain check box of MQTT Explorer)
mosquitto_pub -r -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"white-list":["01:23:14:55:16:15","4C:65:77:88:9C:79","4C:65:A6:66:3C:79"]}'
# Setting the time between BLE scans and force a scan (available with HA discovery)
If you want to change the time between readings you can change the interval by MQTT. For example, if you want the BLE to scan every 66 seconds:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"interval":66000}'
you can also force a scan to be done by the following command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"interval":0}'
TIP
With Home Assistant, this command is directly available through MQTT auto discovery as a switch into the HASS OpenMQTTGateway device entities list.
Once the forced scan has completed, the previous scan interval value will be restored. Forcing a scan command trigger also a BLE connect process after the scan (see below).
The default value TimeBtwRead is set into config_BT.h or into your .ini file for platformio users.
If you want to scan continuously for BLE devices, for example for beacon location you can set the interval to 1ms:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"interval":1}'
In this case you should deactivate the BLE connection mechanism to avoid concurrency between scan and connections (see chapter below, bleconnect).
TIP
For certain devices like LYWSD03MMC OpenMQTTGateway use a connection (due to the fact that the advertized data are encrypted), this connection mechanism is launched after every TimeBtwConnect per default, you can modify it by following the procedure below.
# Setting the time between connection attempts (default: 60min, available with HA discovery)
If you want to change the time between BLE connect you can change it by MQTT, if you want the BLE connect time to be every 300s:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"intervalcnct":300000}'
# Setting if the gateway publishes all the BLE devices scanned or only the detected sensors (default: false, available with HA discovery)
If you want to change this characteristic:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"onlysensors":true}'
The gateway will publish only the detected sensors like Mi Flora, Mi jia, LYWSD03MMC... and not the other BLE devices. This is useful if you don't use the gateway for presence detection but only to retrieve sensors data.
# Setting if the gateway publishes known devices which randomly change their MAC address (default: false)
The default is false, as such changing MAC addresses cannot be related to specific devices.
If you want to change this characteristic:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"randommacs":true}'
# Setting if the gateway use adaptive scanning (default: true, available with HA discovery)
Adaptive scanning lets the gateway decide for you the best passive interval and active intervalacts scan interval, depending on the characteristics of your devices.
The gateway retrieves your devices' information from Theengs Decoder (opens new window) and adapts its parameters accordingly if a device that requires it is detected.
For example a door or a PIR sensor will require continuous scanning, so if detected the gateway is going to reduce its time between scans to the minimum. Or your devices may also require active scanning to retrieve data, in this case the gateway will also trigger active scans at regular intervals.
If you want to change this characteristic (default:true):
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"adaptivescan":false}'
Setting Adaptive scanning to false will automatically put the gateway to continuous active scanning if no additional manual changes have already been applied.
An overview with background information to better understand the different setting used:
Passive scanning: With this scanning mode the gateway picks up any freely available broadcasts sent out by devices, without any interaction with the devices. The interval for this is set with {"interval":66000}
Active scanning: With this scanning mode the gateway sends out requests for sensor broadcasts first, before then picking up the broadcast advertisement data. Some devices require this request before they send out all data in their broadcasts. The interval for this active scanning with request first is set by {"intervalacts":300000}
If adaptive scanning is set to false and you want to manually set these intervals, setting Publishing advertisement and advanced data to true will show you additional data about which of your devices require active scanning and/or continuous scanning, so that you can tune these setting to your devices and your individual requirements of their data.
"cont":true - the device requires continuous scanning. If passive ({"interval":100}) or active ({"intervalacts":100}) depends on the additional device specification.
"acts":true - the device requires active scanning to broadcast all of it's data for decoding.
# Setting the time between active scanning (available with HA discovery)
If you have passive scanning activated, but also have some devices which require active scanning, this defines the time interval between two intermittent active scans.
If you want to change the time between active scans you can change it by MQTT. For setting the active scan interval time to every 5 minutes:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"intervalacts":300000}'
# Setting the duration of a scan (available with HA discovery)
If you want to change the default duration of each scan cycle to 5 seconds
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"scanduration":5000}'
# Setting if the gateway connects to BLE devices eligibles (default: true, available with HA discovery)
If you want to change this characteristic:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"bleconnect":false}'
# Setting if the gateway publish into Home Assistant Home presence topic (default: false, available with HA discovery)
If you want to publish to Home Assistant presence topic, you can activate this function by the HASS interface (this command is auto discovered), here is a yaml example. Or by an MQTT command.
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"hasspresence":true}'
To change presence publication topic, use this MQTT command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"prestopic":"presence/"}'
To use iBeacon UUID for presence, instead of sender (random) MAC address, use this MQTT command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"presuseuuid":true}'
This will change usual payload for iBeacon from:
{"id":"60:87:57:4C:9B:C2","mac_type":1,"rssi":-78,"distance":7.85288,"brand":"GENERIC","model":"iBeacon","model_id":"IBEACON","mfid":"4c00","uuid":"1de4b189115e45f6b44e509352269977","major":0,"minor":0,"txpower":-66}
To:
{"id":"1de4b189115e45f6b44e509352269977","mac_type":1,"rssi":-78,"distance":7.85288,"brand":"GENERIC","model":"iBeacon","model_id":"IBEACON","mfid":"4c00","uuid":"1de4b189115e45f6b44e509352269977","major":0,"minor":0,"txpower":-66,"mac":"60:87:57:4C:9B:C2"}
Note: the MAC address is put in "mac" field.
# Setting if the gateway uses iBeacon UUID as topic, instead of (random) MAC address (default: false)
By default, iBeacon are published like other devices, using a topic based on the MAC address of the sender. But modern phones randomize their Bluetooth MAC address making it difficult to track iBeacon.
For example, the 2 following messages corresponds to the same iBeacon, but with different MAC and topics:
home/OpenMQTTGateway/BTtoMQTT/58782076BC24 {"id":"58:78:20:76:BC:24","mac_type":1,"rssi":-79,"brand":"GENERIC","model":"iBeacon","model_id":"IBEACON","mfid":"4c00","uuid":"1de4b189115e45f6b44e509352269977","major":0,"minor":0,"txpower":-66}
home/OpenMQTTGateway/BTtoMQTT/5210A84690AC {"id":"52:10:A8:46:90:AC","mac_type":1,"rssi":-77,"brand":"GENERIC","model":"iBeacon","model_id":"IBEACON","mfid":"4c00","uuid":"1de4b189115e45f6b44e509352269977","major":0,"minor":0,"txpower":-66}
2
To use iBeacon UUID as topic, use this MQTT command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubuuid4topic":true}'
Resulting in such messages (for the same iBeacon as previously):
home/OpenMQTTGateway/BTtoMQTT/1de4b189115e45f6b44e509352269977 {"id":"52:10:A8:46:90:AC","mac_type":1,"rssi":-76,"brand":"GENERIC","model":"iBeacon","model_id":"IBEACON","mfid":"4c00","uuid":"1de4b189115e45f6b44e509352269977","major":0,"minor":0,"txpower":-66}
home/OpenMQTTGateway/BTtoMQTT/1de4b189115e45f6b44e509352269977 {"id":"7B:63:C6:82:DC:57","mac_type":1,"rssi":-83,"brand":"GENERIC","model":"iBeacon","model_id":"IBEACON","mfid":"4c00","uuid":"1de4b189115e45f6b44e509352269977","major":0,"minor":0,"txpower":-66}
2
# Setting the minimum RSSI accepted to publish device data (default:-100)
If you want to change the minimum RSSI value accepted for a device to be published, you can change it by MQTT. For example if you want to set -80
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"minrssi":-80}'
you can also accept all the devices by the following command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"minrssi":-200}'
The default value is set into config_BT.h
# Store BLE configuration into the gateway
OpenMQTTGateway has the capability to save the current configuration and reload it at startup.
To store the running configuration into the gateway, use the command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"save":true}'
At any time, you can reload the stored configuration with the command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"load":true}'
If you want to erase the stored configuration, use the command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"erase":true}'
Note that it will not change the running configuration, only ensure default configuration is used at next startup.
By the way, if you want to load the default built-in configuration (on any board, not only ESP32), use the command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"init":true}'
Note that it will not change the stored configuration, erase or save is still needed to overwrite the saved configuration.
# Read/write BLE characteristics over MQTT
The gateway can read and write BLE characteristics from devices and provide the results in an MQTT message.
TIP
These actions will be taken on the next BLE connection, which occurs after scanning and after the scan count is reached, see above to set this.
This can be overridden by providing an (optional) parameter "immediate": true within the command. This will cause the BLE scan to stop if currently in progress, allowing the command to be immediately processed. All other connection commands in queue will also be processed for the same device, commands for other devices will be deferred until the next normally scheduled connection.
Note Some devices need to have the MAC address type specified. You can find this type by checking the log/MQTT data and looking for "mac_type". The mac_type of your device can be seen by setting pubadvdata to true with an MQTT command (see Publishing advertisement data), or with the macro pubBLEAdvData true. By default the type is 0 but some devices use different type values. You must specify the correct type to connect successfully.
To specify the MAC address type add the parameter "mac_type" to the command. For example "mac_type": 1 to connect with a device with the MAC address type of 1.
# Example write command
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT -m '{
"ble_write_address":"AA:BB:CC:DD:EE:FF",
"ble_write_service":"cba20d00-224d-11e6-9fb8-0002a5d5c51b",
"ble_write_char":"cba20002-224d-11e6-9fb8-0002a5d5c51b",
"ble_write_value":"TEST",
"value_type":"STRING",
"ttl":4,
"immediate":true }'
2
3
4
5
6
7
8
Response:
{
"id":"AA:BB:CC:DD:EE:FF",
"service":"cba20d00-224d-11e6-9fb8-0002a5d5c51b",
"characteristic":"cba20002-224d-11e6-9fb8-0002a5d5c51b",
"write":"TEST",
"success":true
}
2
3
4
5
6
7
# Example read command
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT -m '{
"ble_read_address":"AA:BB:CC:DD:EE:FF",
"ble_read_service":"cba20d00-224d-11e6-9fb8-0002a5d5c51b",
"ble_read_char":"cba20002-224d-11e6-9fb8-0002a5d5c51b",
"value_type":"STRING",
"ttl": 2 }'
2
3
4
5
6
Response:
{
"id":"AA:BB:CC:DD:EE:FF",
"service":"cba20d00-224d-11e6-9fb8-0002a5d5c51b",
"characteristic":"cba20002-224d-11e6-9fb8-0002a5d5c51b",
"read":"TEST",
"success":true
}
2
3
4
5
6
7
TIP
The ttl parameter is the number of attempts to connect (defaults to 1), which occur after the BLE scan completes.
value_type can be one of: STRING, HEX, INT, FLOAT. Default is STRING if omitted in the message.
# SwitchBot Bot control (available with HA discovery)
SwitchBot Bot devices are automatically discovered and available as a device in the configuration menu of home assistant.
TIP
If the SwitchBot mode is changed the ESP32 must be restarted.
The device can also be controlled over MQTT with a simplified BLE write command.
# Example command to set the SwitchBot state to ON:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT -m '{
"SBS1":"on",
"mac":"AA:BB:CC:DD:EE:FF"
}'
2
3
4
Response (assuming success):
{
"id":"AA:BB:CC:DD:EE:FF",
"state":"on"
}
2
3
4
# ADVANCED: Setting up an external decoder
This advanced option is used to publish raw radio frames on a specific topic to be decoded by an external decoder instead of the integrated one.
To enable external decoder:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"extDecoderEnable":true}'
To change the default external decoder topic to "undecoded":
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"extDecoderTopic":"undecoded"}'
# ADVANCED: Filtering out connectable devices (default: false)
With OpenHAB integration, this configuration is highly recommended, otherwise you may encounter incomplete data.
If you want to enable this feature:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"filterConnectable":true}'
# ADVANCED: Advertisement and advanced data (default: false, available with HA discovery)
If you want to enable this feature:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubadvdata":true}'
This will publish extensive information about the device:
{"id":"11:22:33:44:55:66","mac_type":0,"adv_type":0,"name":"Qingping Motion & Light","rssi":-93,"servicedata":"88121122334455660201520f0126090403000000","servicedatauuid":"0xfdcd","brand":"Qingping","model":"Motion & Light","model_id":"CGPR1","lux":3,"batt":82}
To stop publishing advertisement data:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"pubadvdata":false}'
Note
All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
WARNING
We strongly encourage the use of a white-list (see below) so as to collect data from your devices only and not from other MAC addresses. By default the gateway scans the advertizing BLE devices nearby with their MAC addresses. Depending on your country, it may be illegal to monitor networks for MAC addresses, especially on networks that you do not own. Please check your country's laws (for US Section 18 U.S. Code § 2511) - discussion here (opens new window).