Ir para o conteúdo

Component: MQTT

Responsibility

Federated communication bus. Carries two types of traffic with distinct semantics:

  1. Federated (fl/) — round configuration, local updates, candidate global model.
  2. IoT (iot/) — observations and events from real devices or replay adapters.

Every message is wrapped in an MqttEnvelope<T> with an asymmetric X.509 or HMAC-SHA256 signature (SignatureService / X509SignatureService / HmacSignatureService / CompositeSignatureService), an anti-replay nonce, and a payload hash.

BL-241/BL-242 REST+SSE migration reverted (BL-259): between BL-241 and this revert, the federated (fl/) traffic described here briefly ran over a second production implementation, RestSseGateway (REST+SSE, module federated-digital-twin-broker), for every JVM/Docker composition root, and — since BL-240/BL-254 — the Android client too. At the user's explicit request that migration was reverted end to end: MqttGateway remains transport-agnostic by design, but every composition root (JVM/Docker and Android) is back to a single PahoMqttGateway instance carrying both the IoT (iot/)/CDDL (mhub/) traffic and the federated (fl/) round protocol — the ingestGateway/round-protocol-gateway split introduced by the migration was removed. Everything on this page — the envelope format, topic strings, signature/integrity invariants — was never affected by either the migration or the revert. RestSseGateway still exists (module federated-digital-twin-broker) but is now used only for the unrelated BL-243 internal microbroker hop inside federated-digital-twin-broker — see 18-rest-sse-gateway.md (historical record) and 17-local-digital-twin-broker.md. See 04-orchestrator.md for ClientUpdateRelay (BL-242, kept — only its transport reverted — the client never publishes directly to the aggregator-facing topic).

Implementation

Item Value
Module (interface, envelope, codec) modules/federated-common
Module (MQTT implementation) modules/federated-containers
Java package br.ufma.lsdi.common.mqtt (interface) / br.ufma.lsdi.mqtt.paho (MQTT)
Interface MqttGateway
Production implementation (iot//mhub/ and fl/, every composition root) PahoMqttGateway (Eclipse Paho MQTTv5) over a real broker
Test fixture (not for production) InMemoryMqttGateway — published in the federated-common test-jar, synchronous delivery with no network
Federated topic generation MqttTopics.*()
IoT topic generation IotTopics.*()

MqttGateway — interface

public interface MqttGateway {
    void publish(String topic, MqttEnvelope<?> envelope);

    // payloadType: concrete class of the payload expected on the topic — needed because
    // PahoMqttGateway receives raw bytes from the broker and needs to know which type to deserialize into.
    void subscribe(String topicFilter, Class<?> payloadType, MqttMessageHandler handler);
    // topicFilter accepts MQTT wildcards (+/#): "fl/experiments/+/rounds/+/config"
}

PahoMqttGateway (production)

The single connection point to a real MQTT broker (Eclipse Paho MQTTv5). Uses a single MqttCallback registered on the Paho client and dispatches messages to the handlers locally via MqttTopicMatcher (instead of the subscribe(...,IMqttMessageListener) overloads from the Paho API itself — client version 1.2.5 has an infinite recursion bug in that overload when used directly).

MQTT_BROKER_URL=ssl://localhost:8883
MQTT_CLIENT_ID=hdt-af-admin            # optional — default: hdt-af-{random uuid}
MQTT_TLS_TRUSTSTORE=deploy/mqtt/tls/mqtt-truststore.p12
MQTT_TLS_TRUSTSTORE_PASSWORD=changeit
MQTT_TLS_KEYSTORE=deploy/mqtt/tls/mqtt-client.p12
MQTT_TLS_KEYSTORE_PASSWORD=changeit

MqttBrokerConfig.fromEnvironment() reads these variables. For ssl://, the PKCS12 truststore and keystore are mandatory: a missing CA or client certificate fails before the connection is made. PahoMqttGateway builds a TLS 1.3 SSLContext with both and does not accept the JVM's global truststore. The Docker Compose setup exposes only 8883, with require_certificate true, allow_anonymous false, and use_identity_as_username true. Generate the local, non-versioned material with:

sh scripts/generate-mqtt-tls.sh
docker compose up -d mosquitto

tcp:// remains accepted only for test fixtures and explicitly configured legacy environments; Compose and .env.example use mTLS by default.

InMemoryMqttGateway (test fixture, not for production)

Delivers messages synchronously, on the same call stack as publish(), with no network. It is no longer a production implementation option — it exists only for unit/integration tests that depend on MqttGateway incidentally (e.g., MqttClientUpdateCollectorTest, ClientDiscoveryServiceTest), avoiding the cost of a real broker via Testcontainers. Published in the federated-common test-jar (classifier=tests); isolates exceptions per handler (a failure in one subscriber does not prevent the others from receiving the message).


MqttEnvelope<T> — message envelope

public record MqttEnvelope<T>(
    String   messageId,       // unique UUID v4
    String   schemaVersion,   // "1.0.0"
    String   experimentId,
    String   contractVersion,
    long     roundId,
    String   senderId,
    Instant  timestamp,
    String   nonce,           // anti-replay UUID v4
    String   payloadHash,     // "sha256:{64 hex}" of the serialized payload
    String   signature,       // "x509:{id}:{base64}" or HMAC-SHA256 hex
    T        payload
) {
    // Creation with signature
    static <T> MqttEnvelope<T> signed(experimentId, contractVersion, roundId,
        senderId, payloadEvidence, payload, signatureService);

    // Verification by the receiver (MUST be called before processing)
    boolean verify(SignatureService signatureService);

    // Verification that the transported payload matches the signed payloadHash.
    // verify() alone only proves that the hash was signed by senderId — never that the
    // hash matches the actual payload. Necessary since the payload started being
    // transported as bytes over a real broker (previously, in memory, this was harmless
    // because payload and hash were produced on the same call stack).
    boolean verifyPayloadIntegrity();
}

Canonical string for signing:

messageId|experimentId|contractVersion|roundId|senderId|timestamp|nonce|payloadHash

EvidenceProducing — prerequisite for verifyPayloadIntegrity()

Payloads that implement br.ufma.lsdi.common.model.EvidenceProducing expose evidenceString() — a deterministic canonical serialization of themselves. verifyPayloadIntegrity() recomputes Hashing.sha256(payload.evidenceString()) and compares it with payloadHash; types that do not implement the interface return true (additive compatibility). Implemented by 11 payload records in br.ufma.lsdi.common.model: RoundConfig, RoundFinalizationSignal, ClientUpdateMessage, GlobalModelPublication, ObservationMessage, EventMessage, ClientAnnouncePayload, ContractPublishedPayload, TwinValidationDecisionMessage, ClientArtifactPayload, and GlobalModelPayload.

Wire format — MqttPayloadCodec

br.ufma.lsdi.common.mqtt.MqttPayloadCodec uses Jackson ObjectMapper (dates in ISO-8601 via JavaTimeModule, serialized in camelCase matching the Java record accessors) to serialize/deserialize the complete MqttEnvelope<T> (metadata + payload) to/from JSON. JsonCodec (br.ufma.lsdi.common.util) and ExperimentPackageDescriptor / ClientPackageDescriptor (federated-experiment-packager) also use Jackson ObjectMapper directly. PahoMqttGateway uses MqttPayloadCodec to publish/consume bytes; the subscriber declares the expected Class<?> payloadType for the topic via subscribe(...) so the codec knows which type to deserialize into.


Federated topics (fl/)

BL-241/BL-242 had briefly moved these topics onto RestSseGateway (REST+SSE, the topic string reused as the topic/topicFilter query parameter) for every JVM/Docker composition root. That migration was reverted (BL-259): every composition root, JVM/Docker and Android alike, publishes and subscribes these exact strings over real MQTT via PahoMqttGateway again.

Topic Payload Published by
fl/experiments/{expId}/contracts/published ContractPublishedPayload Orchestrator / Admin Monitor
fl/experiments/{expId}/contracts/{contractVersion}/clients/{clientId}/consent ConsentStatusPayload Client / Admin Monitor
fl/experiments/{expId}/rounds/{roundId}/config RoundConfig RoundCoordinator
fl/experiments/{expId}/rounds/{roundId}/finalization RoundFinalizationSignal RoundCoordinator
fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/status ClientRoundStatusPayload FederatedClientService
fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/updates ClientUpdateMessage FederatedClientService (client-facing — BL-242)
fl/experiments/{expId}/rounds/{roundId}/aggregator/clients/{clientId}/updates ClientUpdateMessage (same envelope) ClientUpdateRelay (aggregator-facing — BL-242)
fl/experiments/{expId}/rounds/{roundId}/aggregation/status AggregationStatusPayload Orchestrator / Aggregator
fl/experiments/{expId}/models/{modelVersion}/published GlobalModelPublication AggregationResultPublisher
fl/experiments/{expId}/twin/clients/{clientId}/models/{modelVersion}/decision TwinValidationDecisionMessage DigitalTwinValidatorAdapter
fl/experiments/{expId}/clients/{clientId}/announce ClientAnnouncePayload Client / Participant Daemon
fl/experiments/{expId}/clients/{clientId}/artifacts/{roundId}/{filename} ClientArtifactPayload TwinArtifactChannel
fl/experiments/{expId}/models/{modelVersion}/payload GlobalModelPayload AggregationResultPublisher

Generation via MqttTopics

MqttTopics also exposes wildcard helper constants and methods (CLIENT_ANNOUNCE_WILDCARD, ALL_EXPERIMENT_ROUND_CONFIGS, ALL_EXPERIMENT_ROUND_FINALIZATIONS, ALL_EXPERIMENT_CONTRACT_PUBLICATIONS, CLIENT_UPDATE_WILDCARD, AGGREGATOR_CLIENT_UPDATE_WILDCARD, CLIENT_ARTIFACT_WILDCARD, allRoundConfigs(expId), allRoundFinalizations(expId), allClientUpdates(expId), allClientArtifacts(expId), allGlobalModelPayloads(expId), allTwinValidationDecisions(expId), allModelPublications(expId)), plus device-local MDPL topics (mdplTrainingRequest, mdplTrainingResult, mdplTrainingReady).

String configTopic = MqttTopics.roundConfig(experimentId, roundId);
// "fl/experiments/exp-har-001/rounds/1/config"

String finalizationTopic = MqttTopics.roundFinalization(experimentId, roundId);
// "fl/experiments/exp-har-001/rounds/1/finalization"

String updateTopic = MqttTopics.clientUpdate(experimentId, roundId, clientId);
// "fl/experiments/exp-har-001/rounds/1/clients/3/updates"

String modelTopic = MqttTopics.modelPublished(experimentId, modelVersion);
// "fl/experiments/exp-har-001/models/2/published"

IoT & CDDL topics (iot/ and mhub/)

Topic Payload Published by
iot/clients/{clientId}/entities/{entityId}/observations ObservationMessage IoT Adapters
iot/clients/{clientId}/entities/{entityId}/events EventMessage IoT Adapters
mhub/{clientId}/service_topic/{serviceName} ObservationMessage Replay Adapters (Har, Ecg, Csv) / Android CDDL (BL-237)
microbroker/cddl/event/object/discovered · disconnected EventMessage CDDL Event Publishers

IotTopics provides entityObservation, entityEvent, allObservationsForClient, allEventsForClient, cddlService, cddlServicesForClient, cddlAllServices, cddlEventObjectDiscovered, cddlEventObjectDisconnected, and cddlAllEvents.

ObservationMessage

public record ObservationMessage(
    String  entityId,
    String  entityType,      // "ecg_heartbeat", "smartphone_activity_window", "equipment"
    String  propertyCode,    // "hr", "f001_tBodyAcc-mean()-X", "temperature"
    String  propertySystem,  // "uci-har", "scada", "ecg-sensor"
    Double  valueNumeric,
    String  valueText,
    String  unitUcum,
    Instant observedAt,
    double  qualityScore     // 0.0 – 1.0
) {
    String evidenceString(); // for hashing the payloadHash
}

EventMessage

public record EventMessage(
    String  entityId,
    String  entityType,
    String  eventTypeCode,   // "equipment_failure", "ecg_anomaly"
    String  eventTypeSystem,
    Instant eventStart,
    String  severity         // "high", "INFO"
) {}

Federated payloads

RoundConfig

Published by the orchestrator to start a round. Clients filter by selectedClients.

public record RoundConfig(
    String   experimentId,
    String   contractVersion,
    long     roundId,
    long     globalModelVersion,
    RoundPhase phase,
    String   globalModelHash,          // hash of the previous model (FedProx)
    String   globalModelUri,           // off-chain URI of the previous model
    Map<String,String> hyperparameters,// epochs, lr, mu, selected_clients, ...
    Instant  deadline,
    AggregationPolicy  aggregationPolicy,
    PrivacyPolicy      privacyPolicy,
    TimeWindow         dataWindow,
    AlgorithmSelection algorithmSelection  // BL-155 remote algorithm/version negotiation
) {
    // 12-arg backward-compatibility constructor (defaults algorithmSelection to unspecified),
    // hasGlobalModel(), expectedUpdates(int), minimumExpectedUpdates(int), evidenceString()
}

ClientUpdateMessage

Published by the client after training. Contains an off-chain URI, never the weight vector.

public record ClientUpdateMessage(
    String experimentId,
    String contractVersion,
    long   roundId,
    String clientId,
    String updateUri,     // "file://target/artifacts/.../local_update.bin"
    String updateHash,    // sha256 of the file
    String metricsUri,
    String metricsHash,
    String dataProfileUri,
    String dataProfileHash,
    double updateNorm,
    int    sampleCount,
    int    positiveCount,
    String signature
)

GlobalModelPublication

Published by the aggregator after each aggregation round.

public record GlobalModelPublication(
    String experimentId,
    String contractVersion,
    long   roundId,
    long   modelVersion,
    String modelUri,    // off-chain URI of the candidate global model
    String modelHash,
    String aggregationReportUri,
    String aggregationReportHash,
    String promotionDecision
)

Publish and consume flow

Orchestrator → MqttTopics.roundConfig(expId, roundId) ──────────────────┐
                                                                          │ RoundConfig (signed)
                                                              ┌───────────▼───────────┐
                                                              │    PahoMqttGateway     │
                                                              │  (real MQTT broker)    │
                                                              └──────────────┬─────────┘
                          Client 1  ◄──────────────────────────────────────┤
                          Client 2  ◄──────────────────────────────────────┤
                          Client N  ◄──────────────────────────────────────┘
                               │
                               │ ClientUpdateMessage (signed) — asynchronous delivery
                               ▼
                    MqttClientUpdateCollector.awaitUpdates(...)
                    (blocks until minimum_expected_updates or deadline —
                     necessary because delivery is no longer synchronous)
                               │
                               ▼
                    FedAvgAggregationService
                               │
                               │ GlobalModelPublication (signed)
                               ▼
                         MqttGateway.publish(modelPublished)

Before the migration to a real broker, MqttClientUpdateCollector.updatesFor(...) was an immediate synchronous read — it only worked because InMemoryMqttGateway.publish() delivered before returning. With asynchronous delivery, awaitUpdates(...) uses wait/notifyAll under the same lock to block until the minimum quorum arrives or the round's deadline (RoundConfig.deadline()) expires.


Invariants

  • MqttEnvelope.signed() — the only envelope creation method; never create one manually.
  • envelope.verify(signatureService) — mandatory before processing any received payload.
  • envelope.verifyPayloadIntegrity() — mandatory alongside verify() for payloads that implement EvidenceProducing, since verify() alone does not prove that the payload matches the signed hash (see the EvidenceProducing section above).
  • Receivers that fail verification must discard silently (log + metric, without propagating the payload).
  • Topic strings must be generated via MqttTopics.*() or IotTopics.*() — never built manually.
  • nonce is a UUID v4 generated at creation time — never reuse it.
  • payloadHash is the SHA-256 of the payload text before the envelope's final serialization.
  • Raw data (signals, features, complete weights) never travels over MQTT topics — only URI + hash.
  • InMemoryMqttGateway is not a production option — it exists only as a test fixture (test-jar of federated-common).

Production configuration (TLS 1.3 / mTLS & DockerMqttEnvironment)

# Production broker (TLS 1.3 + mTLS on port 8883 — see mosquitto.conf)
MQTT_BROKER_URL=ssl://localhost:8883
MQTT_CLIENT_ID=hdt-af-admin            # Paho appends "-" + UUID automatically (BL-115)
MQTT_TLS_TRUSTSTORE=deploy/mqtt/tls/mqtt-truststore.p12
MQTT_TLS_TRUSTSTORE_PASSWORD=changeit
MQTT_TLS_KEYSTORE=deploy/mqtt/tls/mqtt-client.p12
MQTT_TLS_KEYSTORE_PASSWORD=changeit
MQTT_TLS_TRUSTSTORE_TYPE=PKCS12         # optional, default PKCS12
MQTT_TLS_KEYSTORE_TYPE=PKCS12           # optional, default PKCS12
MQTT_USERNAME=
MQTT_PASSWORD=

docker/mosquitto/mosquitto.conf configures only listener 8883 with tls_version tlsv1.3, require_certificate true, use_identity_as_username true, and allow_anonymous false. Plain tcp:// remains accepted only when explicitly configured for local unit/integration fixtures. DockerMqttEnvironment (federated-common) propagates MQTT, mTLS, and SIGNING_* environment variables to Docker containers while rewriting loopback broker URLs to host.docker.internal.

Known Eclipse Paho MQTTv5 1.2.5 bug: the overloads subscribe(String, int, IMqttMessageListener) and subscribe(String[], int[], IMqttMessageListener[]) have an infinite recursion bug (the second method calls itself). PahoMqttGateway uses only subscribe(String, int) (without a listener) plus a single global MqttCallback with manual dispatch via MqttTopicMatcher — do not use the per-subscription listener overloads while this client version is in use.