Component: MQTT¶
Responsibility¶
Federated communication bus. Carries two types of traffic with distinct semantics:
- Federated (
fl/) — round configuration, local updates, candidate global model. - IoT (
iot/) — observations and events from real devices or replay adapters.
Every message is wrapped in an MqttEnvelope<T> with an asymmetric X.509 or HMAC-SHA256 signature (SignatureService / X509SignatureService / HmacSignatureService / CompositeSignatureService), an anti-replay nonce, and a payload hash.
BL-241/BL-242 REST+SSE migration reverted (BL-259): between BL-241 and this revert, the federated (
fl/) traffic described here briefly ran over a second production implementation,RestSseGateway(REST+SSE, modulefederated-digital-twin-broker), for every JVM/Docker composition root, and — since BL-240/BL-254 — the Android client too. At the user's explicit request that migration was reverted end to end:MqttGatewayremains transport-agnostic by design, but every composition root (JVM/Docker and Android) is back to a singlePahoMqttGatewayinstance carrying both the IoT (iot/)/CDDL (mhub/) traffic and the federated (fl/) round protocol — theingestGateway/round-protocol-gateway split introduced by the migration was removed. Everything on this page — the envelope format, topic strings, signature/integrity invariants — was never affected by either the migration or the revert.RestSseGatewaystill exists (modulefederated-digital-twin-broker) but is now used only for the unrelated BL-243 internal microbroker hop insidefederated-digital-twin-broker— see 18-rest-sse-gateway.md (historical record) and 17-local-digital-twin-broker.md. See 04-orchestrator.md forClientUpdateRelay(BL-242, kept — only its transport reverted — the client never publishes directly to the aggregator-facing topic).
Implementation¶
| Item | Value |
|---|---|
| Module (interface, envelope, codec) | modules/federated-common |
| Module (MQTT implementation) | modules/federated-containers |
| Java package | br.ufma.lsdi.common.mqtt (interface) / br.ufma.lsdi.mqtt.paho (MQTT) |
| Interface | MqttGateway |
Production implementation (iot//mhub/ and fl/, every composition root) |
PahoMqttGateway (Eclipse Paho MQTTv5) over a real broker |
| Test fixture (not for production) | InMemoryMqttGateway — published in the federated-common test-jar, synchronous delivery with no network |
| Federated topic generation | MqttTopics.*() |
| IoT topic generation | IotTopics.*() |
MqttGateway — interface¶
public interface MqttGateway {
void publish(String topic, MqttEnvelope<?> envelope);
// payloadType: concrete class of the payload expected on the topic — needed because
// PahoMqttGateway receives raw bytes from the broker and needs to know which type to deserialize into.
void subscribe(String topicFilter, Class<?> payloadType, MqttMessageHandler handler);
// topicFilter accepts MQTT wildcards (+/#): "fl/experiments/+/rounds/+/config"
}
PahoMqttGateway (production)¶
The single connection point to a real MQTT broker (Eclipse Paho MQTTv5). Uses a single
MqttCallback registered on the Paho client and dispatches messages to the handlers
locally via MqttTopicMatcher (instead of the subscribe(...,IMqttMessageListener)
overloads from the Paho API itself — client version 1.2.5 has an infinite recursion bug
in that overload when used directly).
MQTT_BROKER_URL=ssl://localhost:8883
MQTT_CLIENT_ID=hdt-af-admin # optional — default: hdt-af-{random uuid}
MQTT_TLS_TRUSTSTORE=deploy/mqtt/tls/mqtt-truststore.p12
MQTT_TLS_TRUSTSTORE_PASSWORD=changeit
MQTT_TLS_KEYSTORE=deploy/mqtt/tls/mqtt-client.p12
MQTT_TLS_KEYSTORE_PASSWORD=changeit
MqttBrokerConfig.fromEnvironment() reads these variables. For ssl://, the PKCS12 truststore
and keystore are mandatory: a missing CA or client certificate fails before the connection is
made. PahoMqttGateway builds a TLS 1.3 SSLContext with both and does not accept the JVM's
global truststore. The Docker Compose setup exposes only 8883, with require_certificate true,
allow_anonymous false, and use_identity_as_username true. Generate the local, non-versioned
material with:
tcp:// remains accepted only for test fixtures and explicitly configured legacy environments;
Compose and .env.example use mTLS by default.
InMemoryMqttGateway (test fixture, not for production)¶
Delivers messages synchronously, on the same call stack as publish(), with no network. It is no
longer a production implementation option — it exists only for unit/integration tests that depend
on MqttGateway incidentally (e.g., MqttClientUpdateCollectorTest,
ClientDiscoveryServiceTest), avoiding the cost of a real broker via Testcontainers.
Published in the federated-common test-jar (classifier=tests); isolates exceptions per handler
(a failure in one subscriber does not prevent the others from receiving the message).
MqttEnvelope<T> — message envelope¶
public record MqttEnvelope<T>(
String messageId, // unique UUID v4
String schemaVersion, // "1.0.0"
String experimentId,
String contractVersion,
long roundId,
String senderId,
Instant timestamp,
String nonce, // anti-replay UUID v4
String payloadHash, // "sha256:{64 hex}" of the serialized payload
String signature, // "x509:{id}:{base64}" or HMAC-SHA256 hex
T payload
) {
// Creation with signature
static <T> MqttEnvelope<T> signed(experimentId, contractVersion, roundId,
senderId, payloadEvidence, payload, signatureService);
// Verification by the receiver (MUST be called before processing)
boolean verify(SignatureService signatureService);
// Verification that the transported payload matches the signed payloadHash.
// verify() alone only proves that the hash was signed by senderId — never that the
// hash matches the actual payload. Necessary since the payload started being
// transported as bytes over a real broker (previously, in memory, this was harmless
// because payload and hash were produced on the same call stack).
boolean verifyPayloadIntegrity();
}
Canonical string for signing:
EvidenceProducing — prerequisite for verifyPayloadIntegrity()¶
Payloads that implement br.ufma.lsdi.common.model.EvidenceProducing expose
evidenceString() — a deterministic canonical serialization of themselves.
verifyPayloadIntegrity() recomputes Hashing.sha256(payload.evidenceString()) and compares
it with payloadHash; types that do not implement the interface return true (additive
compatibility). Implemented by 11 payload records in br.ufma.lsdi.common.model:
RoundConfig, RoundFinalizationSignal, ClientUpdateMessage, GlobalModelPublication,
ObservationMessage, EventMessage, ClientAnnouncePayload, ContractPublishedPayload,
TwinValidationDecisionMessage, ClientArtifactPayload, and GlobalModelPayload.
Wire format — MqttPayloadCodec¶
br.ufma.lsdi.common.mqtt.MqttPayloadCodec uses Jackson ObjectMapper (dates in ISO-8601 via
JavaTimeModule, serialized in camelCase matching the Java record accessors) to
serialize/deserialize the complete MqttEnvelope<T> (metadata + payload) to/from JSON.
JsonCodec (br.ufma.lsdi.common.util) and ExperimentPackageDescriptor / ClientPackageDescriptor
(federated-experiment-packager) also use Jackson ObjectMapper directly. PahoMqttGateway uses
MqttPayloadCodec to publish/consume bytes; the subscriber declares the expected Class<?> payloadType
for the topic via subscribe(...) so the codec knows which type to deserialize into.
Federated topics (fl/)¶
BL-241/BL-242 had briefly moved these topics onto
RestSseGateway(REST+SSE, the topic string reused as thetopic/topicFilterquery parameter) for every JVM/Docker composition root. That migration was reverted (BL-259): every composition root, JVM/Docker and Android alike, publishes and subscribes these exact strings over real MQTT viaPahoMqttGatewayagain.
| Topic | Payload | Published by |
|---|---|---|
fl/experiments/{expId}/contracts/published |
ContractPublishedPayload |
Orchestrator / Admin Monitor |
fl/experiments/{expId}/contracts/{contractVersion}/clients/{clientId}/consent |
ConsentStatusPayload |
Client / Admin Monitor |
fl/experiments/{expId}/rounds/{roundId}/config |
RoundConfig |
RoundCoordinator |
fl/experiments/{expId}/rounds/{roundId}/finalization |
RoundFinalizationSignal |
RoundCoordinator |
fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/status |
ClientRoundStatusPayload |
FederatedClientService |
fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/updates |
ClientUpdateMessage |
FederatedClientService (client-facing — BL-242) |
fl/experiments/{expId}/rounds/{roundId}/aggregator/clients/{clientId}/updates |
ClientUpdateMessage (same envelope) |
ClientUpdateRelay (aggregator-facing — BL-242) |
fl/experiments/{expId}/rounds/{roundId}/aggregation/status |
AggregationStatusPayload |
Orchestrator / Aggregator |
fl/experiments/{expId}/models/{modelVersion}/published |
GlobalModelPublication |
AggregationResultPublisher |
fl/experiments/{expId}/twin/clients/{clientId}/models/{modelVersion}/decision |
TwinValidationDecisionMessage |
DigitalTwinValidatorAdapter |
fl/experiments/{expId}/clients/{clientId}/announce |
ClientAnnouncePayload |
Client / Participant Daemon |
fl/experiments/{expId}/clients/{clientId}/artifacts/{roundId}/{filename} |
ClientArtifactPayload |
TwinArtifactChannel |
fl/experiments/{expId}/models/{modelVersion}/payload |
GlobalModelPayload |
AggregationResultPublisher |
Generation via MqttTopics¶
MqttTopics also exposes wildcard helper constants and methods (CLIENT_ANNOUNCE_WILDCARD, ALL_EXPERIMENT_ROUND_CONFIGS, ALL_EXPERIMENT_ROUND_FINALIZATIONS, ALL_EXPERIMENT_CONTRACT_PUBLICATIONS, CLIENT_UPDATE_WILDCARD, AGGREGATOR_CLIENT_UPDATE_WILDCARD, CLIENT_ARTIFACT_WILDCARD, allRoundConfigs(expId), allRoundFinalizations(expId), allClientUpdates(expId), allClientArtifacts(expId), allGlobalModelPayloads(expId), allTwinValidationDecisions(expId), allModelPublications(expId)), plus device-local MDPL topics (mdplTrainingRequest, mdplTrainingResult, mdplTrainingReady).
String configTopic = MqttTopics.roundConfig(experimentId, roundId);
// "fl/experiments/exp-har-001/rounds/1/config"
String finalizationTopic = MqttTopics.roundFinalization(experimentId, roundId);
// "fl/experiments/exp-har-001/rounds/1/finalization"
String updateTopic = MqttTopics.clientUpdate(experimentId, roundId, clientId);
// "fl/experiments/exp-har-001/rounds/1/clients/3/updates"
String modelTopic = MqttTopics.modelPublished(experimentId, modelVersion);
// "fl/experiments/exp-har-001/models/2/published"
IoT & CDDL topics (iot/ and mhub/)¶
| Topic | Payload | Published by |
|---|---|---|
iot/clients/{clientId}/entities/{entityId}/observations |
ObservationMessage |
IoT Adapters |
iot/clients/{clientId}/entities/{entityId}/events |
EventMessage |
IoT Adapters |
mhub/{clientId}/service_topic/{serviceName} |
ObservationMessage |
Replay Adapters (Har, Ecg, Csv) / Android CDDL (BL-237) |
microbroker/cddl/event/object/discovered · disconnected |
EventMessage |
CDDL Event Publishers |
IotTopics provides entityObservation, entityEvent, allObservationsForClient, allEventsForClient, cddlService, cddlServicesForClient, cddlAllServices, cddlEventObjectDiscovered, cddlEventObjectDisconnected, and cddlAllEvents.
ObservationMessage¶
public record ObservationMessage(
String entityId,
String entityType, // "ecg_heartbeat", "smartphone_activity_window", "equipment"
String propertyCode, // "hr", "f001_tBodyAcc-mean()-X", "temperature"
String propertySystem, // "uci-har", "scada", "ecg-sensor"
Double valueNumeric,
String valueText,
String unitUcum,
Instant observedAt,
double qualityScore // 0.0 – 1.0
) {
String evidenceString(); // for hashing the payloadHash
}
EventMessage¶
public record EventMessage(
String entityId,
String entityType,
String eventTypeCode, // "equipment_failure", "ecg_anomaly"
String eventTypeSystem,
Instant eventStart,
String severity // "high", "INFO"
) {}
Federated payloads¶
RoundConfig¶
Published by the orchestrator to start a round. Clients filter by selectedClients.
public record RoundConfig(
String experimentId,
String contractVersion,
long roundId,
long globalModelVersion,
RoundPhase phase,
String globalModelHash, // hash of the previous model (FedProx)
String globalModelUri, // off-chain URI of the previous model
Map<String,String> hyperparameters,// epochs, lr, mu, selected_clients, ...
Instant deadline,
AggregationPolicy aggregationPolicy,
PrivacyPolicy privacyPolicy,
TimeWindow dataWindow,
AlgorithmSelection algorithmSelection // BL-155 remote algorithm/version negotiation
) {
// 12-arg backward-compatibility constructor (defaults algorithmSelection to unspecified),
// hasGlobalModel(), expectedUpdates(int), minimumExpectedUpdates(int), evidenceString()
}
ClientUpdateMessage¶
Published by the client after training. Contains an off-chain URI, never the weight vector.
public record ClientUpdateMessage(
String experimentId,
String contractVersion,
long roundId,
String clientId,
String updateUri, // "file://target/artifacts/.../local_update.bin"
String updateHash, // sha256 of the file
String metricsUri,
String metricsHash,
String dataProfileUri,
String dataProfileHash,
double updateNorm,
int sampleCount,
int positiveCount,
String signature
)
GlobalModelPublication¶
Published by the aggregator after each aggregation round.
public record GlobalModelPublication(
String experimentId,
String contractVersion,
long roundId,
long modelVersion,
String modelUri, // off-chain URI of the candidate global model
String modelHash,
String aggregationReportUri,
String aggregationReportHash,
String promotionDecision
)
Publish and consume flow¶
Orchestrator → MqttTopics.roundConfig(expId, roundId) ──────────────────┐
│ RoundConfig (signed)
┌───────────▼───────────┐
│ PahoMqttGateway │
│ (real MQTT broker) │
└──────────────┬─────────┘
Client 1 ◄──────────────────────────────────────┤
Client 2 ◄──────────────────────────────────────┤
Client N ◄──────────────────────────────────────┘
│
│ ClientUpdateMessage (signed) — asynchronous delivery
▼
MqttClientUpdateCollector.awaitUpdates(...)
(blocks until minimum_expected_updates or deadline —
necessary because delivery is no longer synchronous)
│
▼
FedAvgAggregationService
│
│ GlobalModelPublication (signed)
▼
MqttGateway.publish(modelPublished)
Before the migration to a real broker,
MqttClientUpdateCollector.updatesFor(...)was an immediate synchronous read — it only worked becauseInMemoryMqttGateway.publish()delivered before returning. With asynchronous delivery,awaitUpdates(...)useswait/notifyAllunder the same lock to block until the minimum quorum arrives or the round's deadline (RoundConfig.deadline()) expires.
Invariants¶
MqttEnvelope.signed()— the only envelope creation method; never create one manually.envelope.verify(signatureService)— mandatory before processing any received payload.envelope.verifyPayloadIntegrity()— mandatory alongsideverify()for payloads that implementEvidenceProducing, sinceverify()alone does not prove that the payload matches the signed hash (see theEvidenceProducingsection above).- Receivers that fail verification must discard silently (log + metric, without propagating the payload).
- Topic strings must be generated via
MqttTopics.*()orIotTopics.*()— never built manually. nonceis a UUID v4 generated at creation time — never reuse it.payloadHashis the SHA-256 of the payload text before the envelope's final serialization.- Raw data (signals, features, complete weights) never travels over MQTT topics — only URI + hash.
InMemoryMqttGatewayis not a production option — it exists only as a test fixture (test-jar offederated-common).
Production configuration (TLS 1.3 / mTLS & DockerMqttEnvironment)¶
# Production broker (TLS 1.3 + mTLS on port 8883 — see mosquitto.conf)
MQTT_BROKER_URL=ssl://localhost:8883
MQTT_CLIENT_ID=hdt-af-admin # Paho appends "-" + UUID automatically (BL-115)
MQTT_TLS_TRUSTSTORE=deploy/mqtt/tls/mqtt-truststore.p12
MQTT_TLS_TRUSTSTORE_PASSWORD=changeit
MQTT_TLS_KEYSTORE=deploy/mqtt/tls/mqtt-client.p12
MQTT_TLS_KEYSTORE_PASSWORD=changeit
MQTT_TLS_TRUSTSTORE_TYPE=PKCS12 # optional, default PKCS12
MQTT_TLS_KEYSTORE_TYPE=PKCS12 # optional, default PKCS12
MQTT_USERNAME=
MQTT_PASSWORD=
docker/mosquitto/mosquitto.conf configures only listener 8883 with tls_version tlsv1.3,
require_certificate true, use_identity_as_username true, and allow_anonymous false. Plain
tcp:// remains accepted only when explicitly configured for local unit/integration fixtures.
DockerMqttEnvironment (federated-common) propagates MQTT, mTLS, and SIGNING_* environment
variables to Docker containers while rewriting loopback broker URLs to host.docker.internal.
Known Eclipse Paho MQTTv5 1.2.5 bug: the overloads subscribe(String, int, IMqttMessageListener)
and subscribe(String[], int[], IMqttMessageListener[]) have an infinite recursion bug (the second
method calls itself). PahoMqttGateway uses only subscribe(String, int) (without a listener)
plus a single global MqttCallback with manual dispatch via MqttTopicMatcher — do not use the
per-subscription listener overloads while this client version is in use.