Hello,
We are using Ignition 8.3.3 and MQTT Transmission 5.0.2 with a disk-based history store and rolling history buffer enabled.
Configuration:
- Keep Alive: 60s
- Prune Interval: 120s
- Rolling History Max Age: 120s
- Rolling History Prune Quantity: 1000
We occasionally experience network issues that result in an MQTT disconnect and reconnect. The total outage duration is typically less than one minute and rolling buffer prevents data loss.
My expectation is that the rolling buffer could cause duplicate publications for the period between the actual disconnect and when MQTT Transmission detects the failure (up to ~1.5× Keep Alive). Instead, we sometimes see more than 24 hours of previously published historical data being resent by MQTT transmission after reconnecting.
Example:
- A transmitter contains multiple devices under the same Edge Node.
- After a disconnect lasting less than a minute, one device republishes more than 24 hours of historical data, even though most of it was previously published successfully.
- The issue occurs across multiple transmitters and MQTT server connections.
- It is not always the same topic/device that is affected.
- No warnings or errors are logged.
- No duplicate is sent when rolling buffer is de-activated.
Screenshots attached:
Picture1: Ignition logs showing the history publisher publishing one topic with much larger amount of metrics than expected. Same color = same topic
Picture2 Downstream database showing events from ~23 hours before the disconnect being re-ingested by MQTT transmission after reconnecting. In the picture only one tag is shown for a short time period.
We are handling duplicates downstream.
Is what we are seeing expected/know? Is there a way it can be prevented.