Ir para o conteúdo

Federated Simulation Process via MQTT and Docker Containers

This document describes in detail the federated simulation flow with Docker containers (clients and aggregator), covering dataset delivery via MQTT, local training on clients, update collection, and federated aggregation (FedAvg), with governance registration on Hyperledger Fabric.


1. Flow Overview

Diagrama PlantUML 1

Ver código-fonte PlantUML
    @startuml "Federated Simulation Flow via MQTT and Docker"
    !theme plain
    skinparam shadowing false
    autonumber

    actor "Operator / REST API" as Operador
    participant "Admin Web Server & Orchestrator" as AdminAPI
    participant "MqttDatasetReplaySimulator" as Simulador
    participant "MQTT Broker (Port 1883)" as Broker
    participant "Docker Client (DataComponent & TrainApp)" as ClientDocker
    participant "Aggregator Docker Daemon" as AggregatorDocker
    participant "Hyperledger Fabric Ledger" as Fabric

    == PHASE 1 — Dataset Ingestion and Delivery via MQTT ==
    Operador -> AdminAPI : POST /experiments/exp-mnist-digits-001/run-with-dataset
    AdminAPI -> Simulador : simulateAll(config, datasetPackage)
    loop For each CSV record (784 pixels + label) per Client
        Simulador -> Simulador : Rebuilds CDM observations & signs HMAC Envelope
        Simulador -> Broker : PUBLISH iot/clients/{clientId}/entities/{entityId}/observations (QoS 1)
        Simulador -> Broker : PUBLISH iot/clients/{clientId}/entities/{entityId}/events (QoS 1)
        Broker --> ClientDocker : Deliver MQTT Observations & Events
        ClientDocker -> ClientDocker : Stores raw observations in local timeseries.db
    end
    Simulador --> AdminAPI : Returns ReplaySummary (Data loaded on Clients)

    == PHASE 2 — Round Orchestration and Announcement (RoundConfig) ==
    AdminAPI -> Fabric : FederatedExperimentChaincode:registerTrainingRound(expId, roundId)
    Fabric --> AdminAPI : TxId Confirmed (Ledger Mirror)
    AdminAPI -> Broker : PUBLISH fl/experiments/{expId}/rounds/{roundId}/config
    Broker --> ClientDocker : Deliver RoundConfig (Hyperparameters, Epochs, LearningRate)
    Broker --> AggregatorDocker : Deliver RoundConfig (FedAvg Strategy, Quorum)

    == PHASE 3 — Local Training & Delivery of Local Updates ==
    ClientDocker -> ClientDocker : DataComponent.materializeDatasetFromStore() (Validates MQTT Provenance)
    ClientDocker -> ClientDocker : Contract Validation & Preflight (minimumPositiveExamples)
    ClientDocker -> ClientDocker : Runs Mini-Batch SGD on MLP (784 -> 128 -> 10)
    ClientDocker -> ClientDocker : Serializes weight vector W_local and computes SHA-256
    ClientDocker -> Broker : PUBLISH fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/updates (ClientUpdatePayload)
    Broker --> AggregatorDocker : Deliver ClientUpdatePayload (W_local Vector + Hash)
    ClientDocker -> Broker : PUBLISH fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/status (UPDATE_SUBMITTED)
    Broker --> AdminAPI : Deliver Status (UPDATE_SUBMITTED)

    == PHASE 4 — FedAvg Aggregation & Global Model Distribution ==
    AggregatorDocker -> AggregatorDocker : MqttClientUpdateCollector waits for client quorum
    AggregatorDocker -> AggregatorDocker : Runs FedAvg Weighted Average of the weight vectors
    AggregatorDocker -> AggregatorDocker : Generates new candidate Global Model W_global
    AggregatorDocker -> Broker : PUBLISH fl/experiments/{expId}/rounds/{roundId}/aggregation/status (Global Model)
    Broker --> ClientDocker : Deliver Global Model W_global (Base Model Update)
    AggregatorDocker -> AdminAPI : POST /rounds/{roundId}/aggregation-result (Metrics & Hashes)
    AdminAPI -> Fabric : registerAggregationHash(expId, roundId, reportHash)
    AdminAPI -> Fabric : registerGlobalModelVersion(expId, modelVersion, globalModelHash)
    Fabric --> AdminAPI : TxId Registered on the Hyperledger Fabric Ledger
    AdminAPI --> Operador : HTTP 303 Redirect / Round Completed

    @enduml

2. Phase Descriptions

Phase 1 — Dataset Ingestion and Delivery via MQTT

  • Trigger: Fired via the REST request POST /experiments/{expId}/run-with-dataset with the packaged data package ID.
  • Transmission: MqttDatasetReplaySimulator (via CsvDatasetSimulationSource) reads the mnist_flat.csv file, converts the rows into CDM entities, and signs HMAC envelopes.
  • MQTT Topics Used:
  • Pixel/label observations: iot/clients/{clientId}/entities/{entityId}/observations (QoS 1)
  • Diagnostic events: iot/clients/{clientId}/entities/{entityId}/events (QoS 1)
  • Reception in the Client Container: The DataComponent inside the client's Docker container receives the MQTT envelopes and stores the raw records in its local SQLite database (timeseries.db).

Phase 2 — Round Orchestration and Announcement

  • Ledger Registration: The Orchestrator invokes registerTrainingRound on the Fabric chaincode to create the round's auditability anchor.
  • Configuration Announcement: The Orchestrator publishes the RoundConfigPayload message on the topic:
  • fl/experiments/{expId}/rounds/{roundId}/config
  • Reception: Both the Clients and the Aggregator, running in Docker containers, listen to the round configuration topic.

Phase 3 — Local Training and Delivery of Updates to the Aggregator

  • Materialization and Preflight: The client's DataComponent converts the data from timeseries.db, preserving MQTT provenance, and validates the experiment's preflight.
  • Training: The local model (MLP neural network) runs the local training epochs (SGD).
  • Local Update Publication: Each client publishes its local weight vector on the topic:
  • fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/updates
  • Status Signaling: The client notifies the orchestrator by sending the UPDATE_SUBMITTED status on the topic:
  • fl/experiments/{expId}/rounds/{roundId}/clients/{clientId}/status

Phase 4 — Federated Aggregation and Global Model Distribution

  • Collection and Quorum: The aggregator container (DockerAggregatorDaemonApp) listens for updates on the .../updates topic. The MqttClientUpdateCollector waits for the required participant quorum.
  • Aggregation (FedAvg): The aggregator applies the FedAvg strategy (average weighted by each client's local sample count).
  • Global Model Publication: The aggregator sends the new global model vector W_global on the topic:
  • fl/experiments/{expId}/rounds/{roundId}/aggregation/status
  • Reception at the Client: The client containers capture the global model message and update their base model for subsequent rounds.
  • On-Chain Governance: The aggregator sends an HTTP callback (POST /rounds/{roundId}/aggregation-result) to the Admin API, which records on Hyperledger Fabric the hash of the aggregation report (registerAggregationHash) and the new global model version (registerGlobalModelVersion).