Ir para o conteúdo

Federated Experiments Guide — HDT-AF

This guide describes the complete life cycle of a federated experiment in the HDT-AF framework: from scientific conception to execution, monitoring, and audit.


Life cycle overview

1. Specification    → specification.md + manifest.yaml
2. Contract         → ExperimentContract (formal experiment contract)
3. Code             → TrainingLocalModel (local algorithm in Java)
4. Dataset          → DataComponent + data/{client_id}/{experiment_id}/
5. Consent          → clients accept the contract
6. Seed             → initial model v0 (seedGlobalModel)
7. Federated round  → preflight → training → update → aggregation → global model
8. Audit            → event ledger + off-chain artifacts

Ergonomic risk with MQTT simulator

ID: exp-ergonomic-risk-001. Classifies aggregated movement windows as safe or at risk. The simulator publishes six normalized attributes of acceleration, rotation, and orientation and the ergonomic_risk label (0/1) via MQTT; training accesses only the authorized view from the DataComponent.

sh scripts/package-experiment.sh exp-ergonomic-risk-001
sh scripts/package-experiment.sh exp-ergonomic-risk-001 --submit --client-ids 1,2,3

The reference CSV is located at research/experiments/exp-ergonomic-risk-001/data/ergonomic_risk.csv. Without this file, the adapter generates deterministic, non-IID synthetic windows for each client.


Part 1 — Structure of an experiment

Location

Each experiment lives at research/experiments/{experiment_id}/:

research/experiments/{experiment_id}/
├── specification.md        ← hypothesis, dataset, model, federated protocol
├── manifest.yaml           ← entrypoint, algorithm, hyperparameters, targets
├── docker.env.example      ← variables expected in the container
└── src/main/java/
    └── br/ufma/lsdi/research/{domain}/
        ├── {Name}TrainingLocalModel.java          ← implements TrainingLocalModel
        ├── {Name}FederatedExperimentApp.java ← orchestrates the full run
        ├── {Name}DatasetLoader.java         ← partitioning and materialization
        └── (other algorithm-specific files)

specification.md

Scientific document with: - Objective and federated hypothesis - Description of the dataset and its properties (size, features, labels) - Per-client partitioning strategy - Local model architecture - Training hyperparameters (epochs, batch size, learning rate) - Experiment contract (quorum, privacy, metrics) - Execution and target compatibility

manifest.yaml

Mandatory, executable technical metadata file. Main fields:

code:
  code_version: "1.0.0"
  package_format: jar
  entrypoint: br.ufma.lsdi.research.meu.MeuTrainingLocalModel
  train_function: execute
  runtime:
    language: java
    version: "17"
  execution_command: "java -cp target/classes br.ufma.lsdi.research.meu.MeuExperimentApp"

ml:
  algorithm_id: federated_mlp_classifier       # must be in MlAlgorithmCatalog
  algorithm_family: multiclass_classification
  update_format: neural_network_weights_csv
  trainable_on_client: true
  supports_federated_update: true
  supported_targets:
    - jvm_server
    - edge_jvm
    - android_java
    - macos_apple_silicon_java

partitioning:
  strategy: subject_based    # or entity_based, time_based, random
  num_clients: 30

docker.env.example

Example variables for execution in a container:

EXPERIMENT_ID=meu-experimento-001
HAR_MAX_CLIENTS=5
HDT_AF_DATA_ROOT=/app/data
HDT_AF_ARTIFACT_ROOT=/app/artifacts
HDT_AF_RUNTIME_ROOT=/app/runtime
HDT_AF_RESEARCH_ROOT=/app/research

Part 2 — Implementing TrainingLocalModel

TrainingLocalModel is the local training algorithm. It is the only piece of code the researcher needs to implement to integrate with the framework.

Interface to implement

package br.ufma.lsdi.experiment.sdk;

public interface TrainingLocalModel {
    /**
     * Declares the algorithm and its supported targets.
     * Validated by the client before accessing the data.
     */
    MlAlgorithmSpec algorithmSpec();

    /**
     * Executes the local training and returns the results.
     *
     * @param context  round metadata, hyperparameters, privacy policy
     * @param outputDir directory where to write local_update.bin, metrics.json, data_profile.json
     */
    TrainingResult execute(TrainingContext context, Path outputDir);
}

Minimal structure example

public class MeuTrainingLocalModel implements TrainingLocalModel {

    @Override
    public MlAlgorithmSpec algorithmSpec() {
        return MlAlgorithmCatalog.FEDERATED_MLP_CLASSIFIER;  // or FEDERATED_CENTROID_CLASSIFIER, etc.
    }

    @Override
    public TrainingResult execute(TrainingContext context, Path outputDir) {
        // 1. Load authorized data (NEVER open files directly)
        List<double[]> features = context.authorizedDataView().features();
        int[] labels = context.authorizedDataView().labels();

        // 2. Train the local model
        double[] weights = train(features, labels, context.hyperparameters());
        double accuracy = calculateAccuracy(weights, features, labels);

        // 3. Serialize the update to local_update.bin
        Path updatePath = outputDir.resolve("local_update.bin");
        String updateCsv = VectorCodec.toCsv(weights);
        Files.writeString(updatePath, updateCsv, StandardCharsets.UTF_8);
        String updateHash = Hashing.sha256(updateCsv);
        double updateNorm = VectorCodec.norm(weights);

        // 4. Write metrics.json
        Path metricsPath = outputDir.resolve("metrics.json");
        // ... write metrics

        // 5. Write data_profile.json
        Path profilePath = outputDir.resolve("data_profile.json");
        // ... write data profile

        // 6. Return TrainingResult with real metrics
        return new TrainingResult(
            updatePath, metricsPath, profilePath,
            updateHash, updateNorm, TrainingStatus.SUCCESS,
            features.size(), positiveCount,
            accuracy, "train_accuracy",
            epochPoints,           // List<EpochPoint> — per-epoch history
            perClassAccuracy       // Map<String, Double> — accuracy per class
        );
    }
}

What NOT to do in TrainingLocalModel

  • Do not open data/{client_id}/{experiment_id}/dataset.csv directly — always use context.authorizedDataView().
  • Do not write files outside outputDir.
  • Do not connect to the network or external services.
  • Do not include Python, gRPC, or non-Java libraries.
  • Do not return an empty update vector or one with NaN/Inf values.

TrainingResult — important fields

Field Type Description
localUpdatePath Path Path to the local_update.bin file
metricsPath Path Path to metrics.json
dataProfilePath Path Path to data_profile.json
updateHash String SHA-256 of the contents of local_update.bin
updateNorm double L2 norm of the update vector
status TrainingStatus SUCCESS, FAILURE, SKIPPED
sampleCount int Total samples used in training
positiveCount int Positive-class samples (binary classification)
localMetric double Real training metric (e.g., accuracy)
localMetricName String Metric name (e.g., "train_accuracy")
trainingHistory List<EpochPoint> Loss, accuracy, and gradient norm per epoch
perClassAccuracy Map<String, Double> Accuracy per class label

EpochPoint — convergence history

public record EpochPoint(int epoch, double loss, double accuracy, double gradientNorm) {}

Automatically propagated to the web monitor, where it is displayed per client on the round details page.


Part 3 — Preparing the data

Per-client data structure

data/
  {client_id}/
    {experiment_id}/
      dataset.csv     ← features + label, one sample per line
      metadata.json   ← origin, record count, notes

metadata.json — mandatory structure

{
  "client_id": "3",
  "experiment_id": "meu-experimento-001",
  "record_count": 1200,
  "source": "UCI HAR Dataset / subject_id=3",
  "schema_version": "1.0.0",
  "notes": "Partitioned by subject. Only windows from subject 3."
}

Materializing per-client data (DatasetLoader)

Loading the dataset is the experiment's responsibility. The loader must:

  1. Read the raw data source (CSV, database, institutional file).
  2. Partition the records by client.
  3. Write dataset.csv and metadata.json under data/{client_id}/{experiment_id}/.
public class MeuDatasetLoader {

    public void materialize(Path rawDataRoot, Path dataRoot, List<String> clientIds) {
        for (String clientId : clientIds) {
            List<String[]> rows = loadForClient(rawDataRoot, clientId);
            Path dir = dataRoot.resolve(clientId).resolve(EXPERIMENT_ID);
            Files.createDirectories(dir);
            writeCsv(dir.resolve("dataset.csv"), rows);
            writeMetadata(dir.resolve("metadata.json"), clientId, rows.size());
        }
    }
}

How DataComponent accesses the data

The DataComponent (in federated-local-cdm) encapsulates data access:

DataComponent dataComponent = new DataComponent(dataRoot);
DigitalTwinDataset dataset = dataComponent.load(clientId, experimentId);

// The FederatedClientService passes the authorized view to the TrainingLocalModel:
// context.authorizedDataView() → derived from the dataset + contract policy

TrainingLocalModel never reads directly from data/ — it only receives the view authorized by the contract.


Part 4 — Defining the experiment contract

The contract is defined programmatically in Java when creating the experimental application. The fields follow the ExperimentContract class.

Main fields

ExperimentContract contract = new ExperimentContract(
    "meu-experimento-001",      // experimentId
    "1.0.0",                    // contractVersion
    "researcher-001",           // coordinatorId
    "Objetivo do experimento",  // objective
    "human_mobility_digital_twin", // digitalTwinType
    "multiclass_classification",   // taskType
    ContractStatus.ACTIVE,
    Instant.now(),              // validFrom
    Instant.now().plusSeconds(86400), // validTo
    Hashing.sha256(evidence),   // contractHash
    codeRegistry,               // registered training code
    dataSchema,                 // authorized data schema
    studyDesign,                // experimental design
    aggregationPolicy,          // quorum, max_client_weight, max_update_norm
    privacyPolicy,              // differential_privacy, prohibited_outputs
    outputPolicy                // allowed files and metrics
);

AggregationPolicy — quorum and robustness

AggregationPolicy policy = new AggregationPolicy(
    "fedavg",    // aggregation strategy
    18,          // minimumClients — minimum quorum (e.g., 60% of 30)
    3600,        // minimumTotalExamples
    0.70,        // minimumClientWeight
    0.01,        // minimumClientWeight (minimum individual weight)
    5000.0,      // maxUpdateNorm — rejects outliers with a norm above this value
    true         // rejectOutlierUpdates — uses a MAD filter for outliers
);

PrivacyPolicy

PrivacyPolicy privacy = new PrivacyPolicy(
    false,  // differentialPrivacy — DP not implemented in the MVP
    1.0,    // epsilon
    0.0,    // delta
    false,  // secureFederatedAggregation
    3       // minimumClientsForAggregation (k-anonymity protection)
);

OutputPolicy — what the client can send

OutputPolicy output = new OutputPolicy(
    Set.of("local_update.bin", "metrics.json", "data_profile.json"), // allowed files
    Set.of("train_accuracy", "sample_count", "update_norm"),          // allowed metrics
    Set.of("raw_records", "identifiers"),                             // prohibited data
    10L * 1024L * 1024L                                               // maximum size (10 MB)
);

Part 5 — Creating the experimental application

The experimental application ({Name}FederatedExperimentApp) orchestrates the full cycle. Follow the pattern of the existing experiments.

Minimal structure

public class MeuFederatedExperimentApp {

    private static final String EXPERIMENT_ID = "meu-experimento-001";

    public static void main(String[] args) throws Exception {
        run(args.length > 0 ? args[0] : null, false);
    }

    public static FederatedExperimentRunResult run(String datasetRoot, boolean silent) throws Exception {
        // 1. Load and partition the dataset
        MeuDatasetLoader loader = new MeuDatasetLoader();
        loader.materialize(Path.of(datasetRoot), ExperimentRuntimePaths.dataRoot(), clientIds);

        // 2. Infrastructure
        LedgerMode ledgerMode = LedgerMode.fromEnvironment();
        FabricCliLedgerMirror fabricMirror = ledgerMode.usesBlockchain()
                ? new FabricCliLedgerMirror(FabricCliConfig.fromEnvironment(ledgerMode))
                : null;
        FederatedExperimentChaincode experimentChaincode = fabricMirror != null
                ? new FederatedExperimentChaincode(fabricMirror)
                : new FederatedExperimentChaincode();
        ExperimentContractChaincode contractChaincode = new ExperimentContractChaincode();
        ClientConsentChaincode consentChaincode = new ClientConsentChaincode(contractChaincode);
        InMemoryMqttGateway mqttGateway = new InMemoryMqttGateway();
        LocalArtifactExperimentService artifactService =
                new LocalArtifactExperimentService(ExperimentRuntimePaths.artifactRoot());

        // 3. Publish the contract and register consents
        ExperimentContract contract = buildContract();
        contractChaincode.createContract(contract);
        for (String clientId : clientIds) {
            consentChaincode.acceptContract(experimentId, contractVersion, clientId, ...);
        }

        // 4. Register the seed model (v0)
        seedGlobalModel(contract, experimentChaincode, artifactService, modelHash, modelUri);

        // 5. Create the orchestrator, clients, and aggregator
        HmacSignatureService signatures = buildSignatures(clientIds);
        RoundCoordinator orchestrator = new RoundCoordinator(
                contract, consentChaincode, experimentChaincode, mqttGateway, signatures);
        List<FederatedClientService> clients = clientIds.stream()
                .map(id -> new FederatedClientService(id, contract, consentChaincode,
                        experimentChaincode, artifactService, new MeuTrainingLocalModel(), mqttGateway,
                        signatures, dataRoot))
                .toList();
        FedAvgAggregationService aggregator = new FedAvgAggregationService(
                contract, experimentChaincode, artifactService, mqttGateway, signatures, "aggregator-001");

        // 6. Run the rounds
        for (int round = 1; round <= NUM_ROUNDS; round++) {
            RoundConfig roundConfig = orchestrator.startRound(round, globalModelVersion);

            List<ClientExecutionResult> results = clients.stream()
                    .map(c -> c.executeRound(roundConfig))
                    .toList();

            AggregationReport report = aggregator.aggregate(roundConfig, results);
        }

        // 7. Register completion
        experimentChaincode.registerTrainingCompletion(experimentId, "SUCCESS", reportHash);
        return buildRunResult(...);
    }
}

Registering the seed model (v0)

Before the first round, the researcher registers the initial weights in the ledger:

private static void seedGlobalModel(
        ExperimentContract contract,
        FederatedExperimentChaincode experimentChaincode,
        LocalArtifactExperimentService artifactService,
        String seededBy
) {
    // Initialize weights (random with a fixed seed, or zeros)
    double[] initialWeights = initializeWeights(SEED);
    String weightsCsv = VectorCodec.toCsv(initialWeights);
    String modelHash = Hashing.sha256(weightsCsv);

    // Write the off-chain artifact
    ArtifactReference modelRef = artifactService.storeModel(
            contract.experimentId(), 0, weightsCsv);

    // Register on the ledger (v0 = seed model)
    experimentChaincode.seedGlobalModel(
            contract.experimentId(),
            modelHash,
            modelRef.uri(),
            seededBy  // researcher identifier
    );
}

Part 6 — Integrating with the Web Monitor

How the monitor detects the experiment

AdminApplicationBootstrap creates sample experiments automatically. To integrate your experiment into the monitor, there are two approaches:

Approach 1 — Executor registered in AdminMonitoringService:

Add an executor in AdminMonitoringService (or AdminExperimentExecutor) for your experiment. The "Start round" button in the monitor will call MeuFederatedExperimentApp.run().

Approach 2 — Independent execution with shared ledger:

Run the experimental application in the same process as the monitor (or in a separate process pointing to the same persisted ledger). Events registered in FederatedExperimentChaincode are automatically visible in the monitor.

Data visible in the monitor after execution

After each round, the monitor automatically displays (with auto-refresh every 30 s):

  • Dashboard — count of acceptances, rounds, tasks, and assets per experiment
  • Rounds — status, accepted clients, examples, weighted metric, trend (↑/—/↓), average norm
  • Learning metrics — bar history per round
  • Tasks — rank, type, worker, status, diagnostic per task
  • Performances — time series of metrics per client/worker/round
  • Global models — hash, URI, and transaction of each model version
  • Hyperledger assets — evidence, consents, and audit trail on the ledger
  • Per-client convergence — loss/accuracy per epoch + per-class accuracy (HAR)

Part 7 — Registering with Hyperledger Fabric (optional)

Fabric is required for experiment auditing; there is no supported memory-only or hybrid mode. Start the monitor with its Gateway identity and Fabric configuration loaded:

source deploy/fabric/fabric-env.sh
sh scripts/run-admin-web.sh

The monitor uses the Fabric Gateway SDK. Some standalone research-session paths still contain legacy FabricCliLedgerMirror wiring; check the current Fabric integration notes before running those entry points.

Idempotent transactions: if the record already exists in Fabric with the same values, the operation is accepted silently (no exception).


Quick references

Document What it covers
docs/execution-guide.md Scripts, environment variables, generated outputs
docs/components/01-admin-api-monitor-web.md Web monitor resources, routes, JSON API
docs/components/09-experiment-sdk.md TrainingLocalModel, TrainingResult interface, targets
docs/components/10-training-local-model.md TrainingLocalModel rules and invariants
docs/components/06-federated-client.md Client cycle, preflight, update validation
docs/components/11-aggregator.md FedAvg, quorum, outlier rejection
docs/blockchain/02-chaincodes.md Operations and invariants of each Fabric chaincode
research/experiments/exp-har-smartphones-001/specification.md Full example — HAR experiment