victron-esp32-ble-mqtt/README.md

17 KiB
Raw Blame History

🔋 Victron SmartSolar BLE → MQTT

"A Raspberry Pi 4 drawing 4W to read one MPPT is an absurd tax on a solar battery."

ESP32 firmware that reads a Victron SmartSolar MPPT over Bluetooth LE, decrypts the advertisement, and publishes it to MQTT with full Home Assistant auto-discovery — at ~0.15W instead of ~4W.

Licence: MIT Board Built with Crypto MQTT Home Assistant Idle draw Buy me a coffee


The problem

Victron's victron-ble Python library is excellent, and running it on a Raspberry Pi 4 works perfectly — until you notice the Pi is idling at 3–5W purely to relay a few numbers off a charge controller. On an off-grid system that's around 40Wh a night, or 3–4Ah at 12V: the monitoring was costing more than the thing it was monitoring.

This firmware does the same job on an ESP32 at roughly a fortieth of the power, and behaves better in Home Assistant while it's there.

What you get

📶 Reads the advertisement Passive BLE scan of the SmartSolar MPPT's broadcasts — no pairing, no connection, lower draw.
🔐 Real decryption AES-128-CTR exactly as the Victron BLE protocol specifies, matching the reference Python implementation bit for bit.
📡 MQTT every 10s One JSON payload with every value, plus a retained online/offline LWT topic and a diag topic with uptime, RSSI, heap and reconnect counters.
🏠 Auto-discovery Appears in Home Assistant as a Victron SmartSolar MPPT device with entities grouped. No YAML.
🧵 Two cores BLE scanning pinned to core 1, WiFi/MQTT to core 0 via FreeRTOS. Coexistence is handled by a 50% BLE scan duty cycle, not by the core split.
🔁 Self-healing Non-blocking reconnects, forced MQTT reconnect after 5 minutes without a successful publish, and a reboot after 60 minutes
🔌 External load too The MPPT's load output current is published as amps, and multiplied by battery voltage to give watts — so the dashboard shows draw as well as charge.
📊 Dashboard included A ready-made Mushroom + Mini Graph dashboard card and optional template sensors.
🪫 0.15W idle The entire point.

How it fits together

flowchart LR
    mppt["☀️ Victron SmartSolar MPPT<br/><i>BLE advertisement, encrypted</i>"]

    subgraph esp["⚡ ESP32 · dual core"]
        core1["Core 1 · NimBLE<br/><i>passive scan</i>"]
        dec["victron_ble.cpp<br/><i>AES-128-CTR + bit unpack</i>"]
        core0["Core 0 · WiFi + MQTT<br/><i>PubSubClient</i>"]
        core1 --> dec --> core0
    end

    broker["Mosquitto<br/><i>solar/mppt/#</i>"]
    ha["Home Assistant<br/><i>auto-discovered</i>"]

    mppt -.->|"~every few seconds"| core1
    core0 -->|"every 10s"| broker --> ha

    style core1 fill:#e7352c,stroke:#e7352c,color:#fff
    style dec fill:#6f42c1,stroke:#6f42c1,color:#fff
    style core0 fill:#334155,stroke:#334155,color:#fff
    style broker fill:#660066,stroke:#660066,color:#fff
    style ha fill:#41bdf5,stroke:#41bdf5,color:#fff

SmartSolar MPPTs broadcast encrypted BLE advertisements every few seconds. Decryption uses AES-128-CTR with your device's 32-character hex key, and a 16-bit nonce from the advertisement used as the little-endian initial value of a 128-bit counter. The plaintext is a bit-packed structure read LSB-first.


Hardware

  • An ESP32 dev board — tested on the ESP32 CH340 38-pin
  • Within BLE range of the SmartSolar MPPT (~10m)
  • A WiFi network

The ESP32-C3 SuperMini also works, but needs the BOOT button held during flashing. The classic CH340 board is the easier one to live with.

Quick start

1. Clone

git clone https://github.com/MrSossidge/victron-esp32-ble-mqtt.git
cd victron-esp32-ble-mqtt

2. Install PlatformIO

Use either the VS Code extension (handy for interactive work) or the CLI in a Python virtual environment:

  • VS Code — install VS Code and the PlatformIO IDE extension.

  • CLI — PlatformIO needs a Python environment:

    sudo apt install python3-venv          # Debian/Ubuntu, only if missing
    python3 -m venv ~/.venvs/pio
    source ~/.venvs/pio/bin/activate
    python -m pip install --upgrade pip platformio
    pio --version
    

    The venv isolates the pio CLI only; PlatformIO still downloads the ESP32 toolchain and libraries into ~/.platformio (a few hundred MB on the first build).

3. Configure

cp include/example.config.h include/config.h
#define WIFI_SSID       "your_wifi_ssid"
#define WIFI_PASSWORD   "your_wifi_password"
#define MQTT_HOST       "192.168.1.x"        // your MQTT broker IP
#define VICTRON_MAC     "xx:xx:xx:xx:xx:xx"  // lowercase, colon-separated
#define VICTRON_ENC_KEY "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"  // 32 hex chars

config.h is gitignored — your key does not end up in a commit.

Optional settings in the same file:

// 1 = disable WiFi modem sleep (default, recommended)
#define WIFI_DISABLE_POWER_SAVE 1

// 1 = publish Home Assistant discovery, 0 = data/status/diag only
#define HA_DISCOVERY_ENABLED 1
#define HA_DISCOVERY_PREFIX  "homeassistant"
#define HA_DEVICE_ID        "victron_smartsolar_garage"
#define HA_UNIQUE_PREFIX    "victron_mppt"
#define HA_DEVICE_NAME      "Victron SmartSolar MPPT"

If you run a second MPPT, give it its own HA_DEVICE_ID and HA_UNIQUE_PREFIX so its entities and discovery topics do not collide with the first one.

Finding your MAC address and encryption key
  1. Open the Victron Connect app
  2. Connect to your SmartSolar MPPT
  3. Tap the ⋮ menu → Product Info
  4. The MAC address is at the top of that page
  5. Scroll to Instant readout via Bluetooth and tap SHOW for the encryption key

4. Build and upload

With the venv active (or call ~/.venvs/pio/bin/pio directly):

pio run                        # compile only
pio run -t upload              # compile + flash
pio device monitor -b 115200   # serial log at 115200 baud
pio run -t upload -t monitor   # flash, then open the monitor

In VS Code the PlatformIO status bar does the same:

Action Shortcut
Build Ctrl+Alt+B
Upload Ctrl+Alt+U
Serial monitor Ctrl+Alt+S

Build output

Everything is written to the hidden .pio/build/esp32dev/ directory:

.pio/build/esp32dev/
├── firmware.bin      # application image (the file you flash / ship via OTA)
├── bootloader.bin    # ESP32 bootloader
├── partitions.bin    # partition table
├── firmware.elf      # ELF with debug symbols
├── firmware.map      # linker map
└── src/              # compiled objects for main.cpp and victron_ble.cpp
  • .pio/ is gitignored, so build artefacts are never committed.
  • pio run -t upload flashes all three images automatically at the usual offsets: bootloader.bin at 0x1000, partitions.bin at 0x8000, firmware.bin at 0x10000.
  • For OTA updates or a manual esptool write, firmware.bin is the image you need.
  • firmware.elf is used to decode crash backtraces, e.g. xtensa-esp32-elf-addr2line -pfiaC -e .pio/build/esp32dev/firmware.elf <addr>, or with PlatformIO's exception decoder in the serial monitor.

Build notes

  • platformio.ini pins espressif32@6.13.0 (Arduino core 2.0.17). It is the last platform compatible with NimBLE-Arduino 1.4.x — do not bump it without migrating to NimBLE-Arduino 2.x.
  • Dependencies are NimBLE-Arduino 1.4.x, PubSubClient 2.8.x and ArduinoJson 6.21.x, resolved by PlatformIO from lib_deps.
  • A clean build takes a few minutes and needs network access; rebuilds only compile changed files.
  • Reference sizes with the default 4 MB partition table: about 77 % flash and 17 % RAM used. Disabling HA_DISCOVERY_ENABLED frees ~4.6 KB of flash.
  • Clean rebuild: pio run -t clean.

Upload troubleshooting

  • fatal error: config.h: No such file or directory — run step 3 first.
  • Upload times out at 921600 baud — lower upload_speed in platformio.ini to 460800 (or 115200).
  • Permission denied: /dev/ttyUSB0 — sudo usermod -aG dialout $USER, then log out and back in. On Windows, install the CH340 driver.
  • Board not detected — hold BOOT while the upload starts (needed on some ESP32-C3 SuperMini boards).

Manual flashing with esptool

pio run -t upload drives esptool internally. For recovery, a board without PlatformIO, or a machine that only has the pre-built images, flash the three partitions yourself.

pip install esptool

esptool 5.x installs the esptool command and uses hyphenated subcommands (write-flash, erase-flash, merge-bin). The older esptool 4.x used esptool.py with underscores (write_flash); that still works but prints Command 'write_flash' is deprecated, so prefer the 5.x form below.

Port names: Linux /dev/ttyUSB0, macOS /dev/cu.usbserial-XXXX, Windows COM5.

Linux / macOS:

esptool --chip esp32 --port /dev/ttyUSB0 --baud 460800 write-flash \
  0x1000  .pio/build/esp32dev/bootloader.bin \
  0x8000  .pio/build/esp32dev/partitions.bin \
  0x10000 .pio/build/esp32dev/firmware.bin

Windows PowerShell (backtick continues the line):

esptool --chip esp32 --port COM5 --baud 460800 write-flash `
  0x1000  .pio\build\esp32dev\bootloader.bin `
  0x8000  .pio\build\esp32dev\partitions.bin `
  0x10000 .pio\build\esp32dev\firmware.bin

Erase the whole flash first if the board is in a bad state or has a different partition table:

esptool --chip esp32 --port /dev/ttyUSB0 erase-flash

Optional: merge everything into one factory image (handy for archiving or sharing a known-good build):

esptool --chip esp32 merge-bin \
  --flash-mode dio --flash-freq 40m --flash-size 4MB \
  -o factory.bin \
  0x1000  .pio/build/esp32dev/bootloader.bin \
  0x8000  .pio/build/esp32dev/partitions.bin \
  0x10000 .pio/build/esp32dev/firmware.bin

esptool --chip esp32 --port /dev/ttyUSB0 --baud 460800 write-flash 0x0 factory.bin

Notes:

  • These offsets assume the default PlatformIO esp32dev 4 MB layout. If you changed the partition table, flash partitions.bin and firmware.bin using the offsets from that table instead.
  • 921600 baud works on most boards; drop to 460800 or 115200 for flaky CH340 clones.
  • If esptool reports Failed to connect, hold BOOT while it tries, and make sure the serial monitor is closed.
  • firmware.bin alone is what an OTA update expects; the bootloader and partition table are only written over USB.

MQTT topics

Topic Description
solar/mppt/data JSON payload with all sensor values
solar/mppt/status online / offline (retained LWT)
solar/mppt/diag Diagnostics: uptime, WiFi RSSI/status, heap, reconnect counts, BLE packet age
{
  "battery_voltage": "12.98",
  "battery_current": "-0.20",
  "pv_power": "8.0",
  "yield_today_wh": "350",
  "charge_state": "bulk",
  "charge_state_raw": 3,
  "error_code": 0,
  "external_load_a": "0.35",
  "external_load_w": "4.5",
  "data_age_s": 2
}

data_age_s is the age of the last decoded BLE packet. If it keeps growing while solar/mppt/diag still updates, MQTT and WiFi are fine and the problem is BLE reception.

{
  "uptime_s": 86400,
  "wifi_status": 3,
  "wifi_rssi": -61,
  "free_heap": 182344,
  "min_free_heap": 140112,
  "wifi_disconnects": 0,
  "mqtt_connects": 1,
  "publish_failures": 0,
  "ble_valid": 14321,
  "ble_ignored": 0,
  "ble_rssi": -72,
  "ble_age_s": 2,
  "adv_total": 53210,
  "mac_hits": 14400,
  "victron_prefix": 14400,
  "ble_stalls": 0,
  "ble_recoveries": 0
}

BLE counters: adv_total counts every advertisement the scanner delivers, mac_hits those from VICTRON_MAC, and victron_prefix any Victron device. If adv_total rises while mac_hits stays 0, the configured MAC is wrong; the serial log prints the address of the first Victron device it sees. ble_stalls counts scan blocks that failed to complete; ble_recoveries counts NimBLE re-initialisations.

BLE self-healing: after 2 consecutive stalls NimBLE is re-initialised; after 5 the ESP32 reboots. If BLE worked before and no valid packet arrives for 30 minutes, the ESP32 reboots (BLE_STALE_REBOOT_MS).

Charge state values
Value State
0 Off
2 Fault
3 Bulk
4 Absorption
5 Float
7 Equalize
245 Starting
252 External control

Troubleshooting

Start by subscribing to the solar/mppt/diag topic:

Observation Likely cause
diag updates, ble_age_s/data_age_s growing MQTT and WiFi are fine; the MPPT advertisement is not being received (range, MPPT asleep, VictronConnect connected to it, BLE scan failed)
diag stops, status still online WiFi or broker path; watch the serial log for [WiFi] Disconnected, reason= and [MQTT] Connection lost
publish_failures climbing Socket/broker rejects; check broker logs for duplicate MQTT_CLIENT_ID
free_heap low or min_free_heap dropping each boot Memory pressure; shorten the diag interval or reduce scanning

Note that retained online on solar/mppt/status only means the broker has not yet noticed the TCP session is gone; it does not prove the ESP32 is still publishing.

The firmware includes two safety nets (both defined in src/main.cpp):

  • 5 minutes without a successful publish → drop and rebuild the MQTT connection.
  • 60 minutes without any successful publish → ESP.restart().

Home Assistant

Once the ESP32 is running, go to Settings → Devices & Services → MQTT. A Victron SmartSolar MPPT device appears with Battery Voltage (V), Battery Current (A), PV Power (W), Yield Today (Wh), Charge State, Error Code, External Load (A) and External Load Power (W). No manual YAML.

Dashboard card — dashboard.yaml is ready to paste into Dashboard → Edit → Add Card → Manual. It needs two HACS frontend plugins: Mushroom and Mini Graph Card.

Template sensors — configuration_template.yaml adds two optional calculated sensors: Garage Net Battery Power (current × voltage in watts, positive = charging) and Garage Battery Status (Charging / Discharging / Idle). Add the template: block to your configuration.yaml and reload YAML.

Power consumption

Device Idle draw
Raspberry Pi 4 ~3–5 W
ESP32 (this firmware) ~0.15 W

~4W saved overnight is ~40Wh per 10-hour night — roughly 3–4Ah at 12V, which is a real fraction of a small leisure battery.

Layout

├── include/
│   ├── example.config.h         template — copy to config.h
│   └── victron_ble.h            BLE parser header
├── src/
│   ├── main.cpp                 WiFi, MQTT, BLE scanning, FreeRTOS tasks
│   └── victron_ble.cpp          AES decryption and MPPT data parser
├── dashboard.yaml               ready-made Home Assistant dashboard card
├── configuration_template.yaml  optional template sensors
└── platformio.ini               build config

Dependencies

PlatformIO handles all of these:

Licence

MIT.

Support

If this got a Raspberry Pi out of your solar setup, you can buy me a coffee.

Buy Me a Coffee