Ir para o conteúdo

02 — Topics and Payloads

Federated protocol topics use the prefix fl/experiments/{experimentId}/. This document also records the Android MDPL request/result topic names, which are used only by the device-local CDDL microbroker and must not be sent to the remote broker. MqttTopics.java is the source of these topic strings — never build them manually.

IoT topics per experiment (HAR, ECG, DT, pattern for new experiments): see 03-topics-iot-per-experiment.md.

BL-241/BL-242/BL-240/BL-254 REST+SSE migration reverted (BL-259): roundConfig, roundFinalization, clientUpdate, and modelPublished — the round-protocol core — had briefly moved from MQTT to REST+SSE on every JVM/Docker client path, and (since BL-240/BL-254) the Android client too. At the user's explicit request that migration was reverted end to end: every composition root (JVM/Docker and Android alike) publishes/subscribes these topics over plain MQTT again, exactly as the table below always described. The strings/topic shape were never changed by either the migration or the revert — MqttTopics.java never removed them. contractPublished, consentStatus, clientRoundStatus, aggregationStatus, and twinValidationDecision were never migrated and stayed plain MQTT throughout. See docs/components/18-rest-sse-gateway.md for the historical record of the reverted transport.

BL-242 — the client never talks to the aggregator directly (kept, transport reverted). clientUpdate is now "client-facing": the client (JVM or Android) only ever publishes there, and only the orchestrator's new ClientUpdateRelay subscribes to it. The relay republishes the same signed envelope (no re-signing) to a new "aggregator-facing" topic, aggregatorClientUpdate — the only topic MqttClientUpdateCollector subscribes to now. See docs/components/04-orchestrator.md.


Topic map

Archive: the admin monitor can subscribe to MQTT # and persist every received envelope, including fl/, iot/, mhub/ and CDDL event topics. This is an archival observer; it does not change topic routing or replace local DataComponent ingestion. IoT payloads can contain raw observations; see the archive component's privacy and retention requirements.

Topic (pattern) Java method Direction Payload
fl/experiments/{expId}/contracts/published contractPublished(expId) Orchestrator / Monitor → Clients ContractPublishedPayload
fl/experiments/{expId}/clients/{clientId}/consent/status consentStatus(expId, clientId) Client → Orchestrator ConsentStatusPayload
fl/experiments/{expId}/clients/{clientId}/announce clientAnnounce(expId, clientId) Client → Monitor ClientAnnouncePayload
fl/experiments/{expId}/rounds/{roundId}/config roundConfig(expId, roundId) Orchestrator → Clients RoundConfig
fl/experiments/{expId}/rounds/{roundId}/finalization roundFinalization(expId, roundId) Orchestrator → Clients RoundFinalizationSignal
fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/status clientRoundStatus(expId, roundId, clientId) Client → Orchestrator ClientRoundStatusPayload
fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/updates clientUpdate(expId, roundId, clientId) Client → Orchestrator ClientUpdateMessage
fl/experiments/{expId}/rounds/{roundId}/aggregator/clients/{clientId}/updates aggregatorClientUpdate(expId, roundId, clientId) (BL-242) Orchestrator (ClientUpdateRelay) → Aggregator ClientUpdateMessage (same envelope, relayed)
fl/experiments/{expId}/clients/{clientId}/artifacts/{roundId}/{filename} clientArtifact(expId, clientId, roundId, filename) Local Twin / Client → Orchestrator ClientArtifactPayload
fl/experiments/{expId}/rounds/{roundId}/aggregation/status aggregationStatus(expId, roundId) Orchestrator / Aggregator → Clients AggregationStatusPayload
fl/experiments/{expId}/models/{modelVersion}/published modelPublished(expId, modelVersion) Orchestrator → Clients GlobalModelPublication
fl/experiments/{expId}/models/{modelVersion}/payload globalModelPayload(expId, modelVersion) Orchestrator → Local Twin GlobalModelPayload
fl/experiments/{expId}/models/{modelVersion}/clients/{clientId}/twin-validation twinValidationDecision(expId, modelVersion, clientId) Local digital twin → Orchestrator/monitor TwinValidationDecisionMessage

Location: modules/federated-common/src/main/java/br/ufma/lsdi/common/mqtt/MqttTopics.java

Wildcard builders (BL-238 / BL-242): allRoundConfigs(expId), allRoundFinalizations(expId), allModelPublications(expId), allClientUpdates(expId), allClientArtifacts(expId), allGlobalModelPayloads(expId), allTwinValidationDecisions(expId), CLIENT_ANNOUNCE_WILDCARD, and CLIENT_ARTIFACT_WILDCARD are convenience builders/constants over the topic templates above — used by DigitalTwinBridge (federated-digital-twin-broker), ClientArtifactIngestor, TwinArtifactChannel, and AdminWebServer.

Wildcard builders (BL-242): AGGREGATOR_CLIENT_UPDATE_WILDCARD (fl/experiments/+/rounds/+/aggregator/clients/+/updates) is the single topic filter MqttClientUpdateCollector subscribes to, cross-experiment. CLIENT_UPDATE_WILDCARD (fl/experiments/+/rounds/+/clients/+/updates) is the client-facing counterpart — used by ClientUpdateRelay.forAllExperiments(...) (the AdminWebServer cross-experiment relay, an additional defensive layer never depended upon by any *FederatedSession/daemon, each of which already runs its own ClientUpdateRelay.forExperiment(...)).

Android multi-experiment filters: the single foreground app service uses these filters to receive round events for all experiments. The service creates a separate lifecycle per (experimentId, contractVersion) and ignores round configs for which this installation has no matching consent.

Filter constant Topic filter Purpose
ALL_EXPERIMENT_ROUND_CONFIGS fl/experiments/+/rounds/+/config Receive signed round configurations
ALL_EXPERIMENT_ROUND_FINALIZATIONS fl/experiments/+/rounds/+/finalization Receive signed finalization signals
ALL_EXPERIMENT_CONTRACT_PUBLICATIONS fl/experiments/+/contracts/published Refresh the local experiment catalog

These are subscriptions, not publication topics. Android only announces presence for consented experiments; sensor acquisition is shared, while stored observations and pending round state are scoped to the experiment and contract version. See 06-federated-client.md.

Device-local MDPL topics

The following topic names are used only by CDDL on the Android device to carry a verified training work item to the preinstalled MDPL OnArrivalPhenotypeInferenceEngine and return its result. They are not subscribed or published on the remote federated MQTT broker and do not use MqttEnvelope.

Topic pattern MqttTopics method Message
mdpl/clients/{clientId}/training/requests mdplTrainingRequest(clientId) JSON MdplTrainingWorkItem inside local Message.serviceValue[0]
mdpl/clients/{clientId}/training/results mdplTrainingResult(clientId) JSON MdplTrainingResult inside local Message.serviceValue[0]
mdpl/clients/{clientId}/training/ready mdplTrainingReady(clientId) Local client ID string, emitted after the engine subscribes to requests

The request/result bridge validates request identity and waits for the matching result. The training data and these CDDL messages remain on-device; the normal signed ClientUpdateMessage continues over the client-facing fl/ topic after training.


Detailed payloads

ContractPublishedPayload (BL-252, implemented)

Reconciled against the real implementation — this section used to describe an aspirational shape (contract_hash/contract_uri/status/valid_from/valid_to) that was never actually built; the topic itself (MqttTopics.contractPublished(experimentId)) sat completely dead — no publisher, no subscriber — until a user question while reviewing 08-consentimento-usuario.puml ("shouldn't the núcleo federado broadcast a new-contract message to all clients first?") led to actually wiring it up. AdminWebServer.broadcastContractPublished(...) publishes it right after publishExperiment(...) (both the HTML and JSON API routes), signed as "admin-monitor" (fixed shared secret, same convention as "orchestrator-001"/"aggregator-001", BL-234). Deliberately lightweight — it only signals that a contract exists at that version; the receiver still fetches/ verifies the full ExperimentContract via chaincode before deciding consent, same boundary the pre-existing flow already had.

public record ContractPublishedPayload(
        String experimentId,
        String contractVersion,
        Instant publishedAt
) implements EvidenceProducing

Client side (Docker): DockerErgonomicClientDaemonApp.awaitContractPublished(...) subscribes on plain MQTT and blocks — with a bounded timeout, CONTRACT_WAIT_TIMEOUT_SECONDS, default 300s, best-effort — before attempting autoAcceptConsent(...). Before this, the daemon announced and auto-accepted immediately on boot, with no signal that the operator had actually published/activated the contract on the dashboard yet.

Client side (Android, BL-254 and multi-experiment update): FederatedMqttService subscribes to ALL_EXPERIMENT_CONTRACT_PUBLICATIONS on the same remote MQTT connection as the round protocol. ContractPublishedGate verifies the signature and payload integrity and requires the payload experiment ID to match the envelope identity before refreshing the catalog. Android never auto-accepts: the broadcast only triggers MainActivity to rediscover active contracts; acceptance still requires an explicit user decision for each (experimentId, contractVersion).


ConsentStatusPayload

Published by the client after recording acceptance or rejection on the ledger.

{
  "client_id": "client-org-001",
  "status": "accepted",
  "accepted_contract_hash": "sha256:...",
  "fabric_transaction_id": "tx123"
}

Values of status: accepted · rejected · revoked


RoundConfig

Published by RoundCoordinator when starting a new round (MqttPayloadCodec serializes Java records in camelCase).

{
  "experimentId": "exp-har-smartphones-001",
  "contractVersion": "1.0.0",
  "roundId": 3,
  "globalModelVersion": 2,
  "phase": "ACTIVE",
  "globalModelHash": "sha256:...",
  "globalModelUri": "file://target/artifacts/models/v2/model.bin",
  "hyperparameters": {
    "epochs": "18",
    "learning_rate": "0.001",
    "selected_clients": "1,2,3"
  },
  "deadline": "2026-05-27T15:00:00Z",
  "aggregationPolicy": {},
  "privacyPolicy": {},
  "dataWindow": {},
  "algorithmSelection": {
    "algorithmId": "federated_mlp_classifier",
    "algorithmVersion": "1"
  }
}

Java representation: RoundConfig (13 fields including AlgorithmSelection).

RoundFinalizationSignal (BL-082)

Published by RoundCoordinator to trigger local update publication (MqttTopics.roundFinalization(expId, roundId)):

public record RoundFinalizationSignal(
    String experimentId,
    String contractVersion,
    long roundId,
    RoundFinalizationTrigger trigger, // MANUAL | DEADLINE_EXPIRED | ALL_EXPECTED_RECEIVED
    Instant triggeredAt
) implements EvidenceProducing

ClientRoundStatusPayload

Published by the client to report progress within a round.

{
  "client_id": "client-org-001",
  "status": "preflight_passed",
  "preflight_hash": "sha256:...",
  "details_uri": "https://artifacts/client-org-001/round-3/preflight.json"
}

Values of status (in flow order):

Value Meaning
download_started Started downloading the training JAR
package_verified JAR hash verified
preflight_failed Statistical preflight failed — client does not train
preflight_passed Preflight passed — client will start training
training_started TrainingLocalModel.execute() started
training_failed Exception during training
completed_not_submitted Training completed but update not submitted (e.g.: norm above the limit)
update_submitted Update successfully published

ClientUpdatePayload

Published by the client after successful local training, to the client-facing clientUpdate topic. Contains only metadata — the update vector stays off-chain. BL-242: the client never publishes to the aggregator-facing topic — the orchestrator's ClientUpdateRelay republishes the same signed envelope to aggregatorClientUpdate for MqttClientUpdateCollector to consume.

{
  "experimentId": "exp-har-smartphones-001",
  "contractVersion": "1.0.0",
  "roundId": 3,
  "clientId": "client-org-001",
  "updateUri": "file://target/artifacts/exp-har-smartphones-001/client-org-001/round-3/local_update.bin",
  "updateHash": "sha256:...",
  "metricsUri": "file://target/artifacts/exp-har-smartphones-001/client-org-001/round-3/metrics.json",
  "metricsHash": "sha256:...",
  "dataProfileUri": "file://target/artifacts/exp-har-smartphones-001/client-org-001/round-3/data_profile.json",
  "dataProfileHash": "sha256:...",
  "updateNorm": 2.31,
  "sampleCount": 12048,
  "positiveCount": 998,
  "signature": "x509:client-org-001:..."
}

Java representation: ClientUpdateMessage. ClientUpdateResolver (federated-orchestrator) resolves the artifact at updateUri, recomputes updateHash, and feeds the vector to FedAvgAggregationService.

ClientArtifactPayload & GlobalModelPayload (TwinArtifactChannel)

Used between TwinArtifactChannel (federated-client) and ClientArtifactIngestor / AggregationResultPublisher (federated-orchestrator) when artifacts traverse MQTT as Base64 payloads with SHA-256 verification:

  • ClientArtifactPayload(experimentId, contractVersion, roundId, clientId, filename, contentBase64, contentHash, publishedAt)
  • GlobalModelPayload(experimentId, contractVersion, roundId, modelVersion, modelUri, modelHash, contentBase64, publishedAt)

AggregationStatusPayload

Published by the aggregator upon completing the aggregation of a round.

{
  "round_id": 3,
  "status": "completed",
  "quorum_satisfied": true,
  "participating_clients": 8,
  "aggregation_report_uri": "https://artifacts/round-3/aggregation_report.json",
  "aggregation_report_hash": "sha256:..."
}

Values of status: completed · quorum_not_reached · failed


GlobalModelPublication

Published by AggregationResultPublisher when a candidate global model becomes available. The model is not automatically promoted — each client validates it locally before updating the digital twin.

{
  "experimentId": "exp-har-smartphones-001",
  "contractVersion": "1.0.0",
  "roundId": 3,
  "modelVersion": 3,
  "modelUri": "file://target/artifacts/exp-har-smartphones-001/models/v3/model.bin",
  "modelHash": "sha256:...",
  "aggregationReportUri": "file://target/artifacts/exp-har-smartphones-001/round-3/aggregation_report.json",
  "aggregationReportHash": "sha256:...",
  "promotionDecision": "CANDIDATE"
}

Java representation: GlobalModelPublication. Published on fl/experiments/{expId}/models/{modelVersion}/published.


TwinValidationDecisionMessage (BL-238)

Published by the local digital twin (DigitalTwinValidatorAdapter, federated-digital-twin-broker) after evaluating a candidate global model against the local baseline and private holdout — see docs/components/17-local-digital-twin-broker.md.

{
  "experimentId": "exp-har-smartphones-001",
  "contractVersion": "1.0.0",
  "modelVersion": 3,
  "clientId": "client-org-001",
  "decision": "PROMOTE",
  "evaluatedMetricName": "accuracy",
  "evaluatedMetricValue": 0.92,
  "baselineModelHash": "sha256:...",
  "candidateModelHash": "sha256:...",
  "reason": "Metrica 0.92 supera baseline 0.80 pela margem de promocao 0.05",
  "decidedAt": "2026-05-27T15:02:00Z"
}

decision values: PROMOTE · REJECT · DEFER. Java representation: TwinValidationDecisionMessage. Published on fl/experiments/{expId}/models/{modelVersion}/clients/{clientId}/twin-validation.


Chronological flow per round

Orchestrator                         Client                            Aggregator
     │                                  │                                 │
     │── contracts/published ──────────►│                                 │
     │                                  │ (acceptance on the ledger)      │
     │◄── clients/{id}/consent/status ──│                                 │
     │                                  │                                 │
     │── rounds/{r}/config ────────────►│                                 │
     │                                  │── preflight ──────────────────► │ (via ledger)
     │◄── rounds/{r}/clients/{id}/status│ (preflight_passed)              │
     │                                  │── training ────────────────────►│ (via ledger)
     │◄── rounds/{r}/clients/{id}/status│ (update_submitted)              │
     │◄── rounds/{r}/clients/{id}/updates│                                │  (client-facing, BL-242)
     │ (ClientUpdateRelay: same envelope,                                 │
     │  no re-signing) ── rounds/{r}/aggregator/clients/{id}/updates ────►│  (aggregator-facing)
     │                                  │                                 │ (downloads artifact)
     │                                  │                                 │ (aggregates + validates quorum)
     │◄── rounds/{r}/aggregation/status ├─────────────────────────────────│
     │                                  │◄── models/{v}/published ────────│
     │                                  │ (validates locally)             │

BL-242: the client never connects to the aggregator — every ClientUpdateMessage is relayed through the orchestrator's ClientUpdateRelay, one hop, no re-signing (the aggregator verifies the original client signature at MqttClientUpdateCollector.onUpdateMessage).



IoT topics — time-series ingestion

The topics below carry sensor/digital-twin data to the local data component. They use the iot/ prefix to distinguish them from the federated fl/ protocol.

Location: modules/federated-common/src/main/java/br/ufma/lsdi/common/mqtt/IotTopics.java

Topic Java method Direction Payload
mhub/{clientId}/service_topic/{serviceName} cddlService(clientId, serviceName) CDDL Sensor/Simulator → Client ObservationMessage
microbroker/cddl/event/object/discovered cddlEventObjectDiscovered() CDDL Event → Client EventMessage
microbroker/cddl/event/object/disconnected cddlEventObjectDisconnected() CDDL Event → Client EventMessage
iot/clients/{clientId}/entities/{entityId}/observations entityObservation(clientId, entityId) Sensor/DT → Client (Canonical) ObservationMessage
iot/clients/{clientId}/entities/{entityId}/events entityEvent(clientId, entityId) Sensor/DT → Client (Canonical) EventMessage

Subscription wildcard helpers in IotTopics: allObservationsForClient(clientId), allEventsForClient(clientId), cddlServicesForClient(clientId), cddlAllServices(), and cddlAllEvents().

ObservationMessage

Published by sensors or digital twins when generating a new reading.

{
  "entityId": "1-equipment-1",
  "entityType": "equipment",
  "propertyCode": "temperature",
  "propertySystem": "data-component",
  "valueNumeric": 73.5,
  "valueText": null,
  "unitUcum": "Cel",
  "observedAt": "2026-05-27T14:22:00Z",
  "qualityScore": 0.98
}

EventMessage

Published when a relevant discrete event occurs (failure, alert, maintenance).

{
  "entityId": "1-equipment-1",
  "entityType": "equipment",
  "eventTypeCode": "equipment_failure",
  "eventTypeSystem": "data-component",
  "eventStart": "2026-05-27T14:00:00Z",
  "severity": "high"
}

Wildcard to subscribe to all data for a client

iot/clients/1/entities/+/observations
iot/clients/1/entities/+/events

Operational analysis of the IoT topics

  • The iot/ topics are separated from the federated fl/ namespace so that sensor ingestion does not mix with contracts, RoundConfig, updates, and model publication.
  • The client's scope sits at the second level (iot/clients/{clientId}/...), which lets an MqttTimeSeriesIngestor subscribe to only the authorized client's namespace.
  • The entity's scope sits at the fourth level (entities/{entityId}), preserving granularity for selective replay and per-client wildcard filters.
  • Observations and events use different suffixes (observations and events), avoiding payload ambiguity and simplifying validation on the receiver side.
  • The dataset replay MQTT simulator reuses exactly these canonical topics via IotTopics.entityObservation() and IotTopics.entityEvent().

IoT → Federated dataset flow

Sensor/DT or simulator          MqttTimeSeriesIngestor          SqliteTimeSeriesStore
    │                                   │                              │
    │── ObservationMessage ────────────►│                              │
    │                                   │── appendObservation ────────►│
    │                                   │                              │ (mqtt_observations)
    │── EventMessage ──────────────────►│                              │
    │                                   │── appendEvent ──────────────►│
    │                                   │                              │ (mqtt_events)
    .                                   .                              .
    .  [accumulation over time]         .                              .
    .                                   .                              .
                                                                       │
DataComponent.loadLocalCdmRepository() ◄──────────────────────────────┤
    │                                                                  │
    ▼                                                                  │
TimeSeriesDatasetBuilder.build()  ◄───────── readObservations() ──────┤
    │                                ◄─────── readEvents() ───────────┤
    │                                ◄─────── readEntities() ─────────┘
    │
    ▼
InMemoryLocalCdmRepository
    │
    ▼
DataComponent.authorizedTrainingView(...) → AuthorizedTrainingViewWriter
    │
    ▼
FederatedClientService.executeRound() → TrainingLocalModel

SqliteTimeSeriesStore (timeseries.db) is the only supported time-series backend (CsvTimeSeriesStore and CSV snapshot fallback from observations_ts.csv/events_ts.csv/entities_ts.csv were removed in BL-146); DataComponent.authorizedTrainingView(...) (BL-245) enforces contract field filtering before training.


Topic filtering (MQTT wildcards)

To subscribe to all client updates of an experiment within a round:

fl/experiments/exp-har-smartphones-001/rounds/3/clients/+/updates

To subscribe to all client updates across all rounds of an experiment:

fl/experiments/exp-har-smartphones-001/rounds/+/clients/+/updates

+ matches one level. # matches zero or more trailing levels.