Ir para o conteúdo

Component: Federated Client

Responsibility

Executes the experiment within the participant's local domain. Validates consent, runs the statistical preflight, trains the local model, and publishes only the authorized update. The admin-monitor JVM path uses isolated Docker containers; the Android client is a separate, on-device execution path described below.

Docker execution paths (persistent daemons vs. ephemeral per-round containers): The admin-monitor Containers tab launches persistent participant daemons (DockerParticipantLaunchService, e.g., DockerErgonomicClientDaemonApp, DockerSmokeClientDaemonApp) that acquire data via MQTT. Separately, FederatedDockerClientExecutor launches one ephemeral container per client per round (DockerHarClientApp, DockerEcgClientApp) using a filtered read-only timeseries.db export. Research *FederatedSession classes (research/experiments/*) construct and run FederatedClientService in-process by design. (Note: FEDERATED_CLIENT_EXECUTION_MODE is an obsolete environment variable not read by current Java code or scripts.)

BL-242 — the client never connects to the aggregator. The client publishes ClientUpdateMessage exactly as before, to the same client-facing topic — nothing here changed. What changed is who is allowed to consume it: only the orchestrator's new ClientUpdateRelay subscribes to that topic now, and only it forwards the (unmodified, still client-signed) envelope to the aggregator. See 04-orchestrator.md.

Implementation

Item Value
Module modules/federated-client
Java package br.ufma.lsdi.client
Service class FederatedClientService
Docker executor FederatedDockerClientExecutor
Docker image hdt-af-client:local
Dockerfile docker/client/Dockerfile

Main classes

FederatedClientService

Executes the complete cycle of a federated round inside the Docker container.

// Complete cycle of a round (called inside the container)
ClientExecutionResult result = client.executeRound(contract, roundConfig, trainingJob);

Internal flow of executeRound():

1. requireContractActive(contract)           → validates ACTIVE status
2. requireConsent(contract)                  → validates acceptance in the chaincode
3. requireSelectedForRound(roundConfig)      → clientId ∈ selectedClients
4. requireRemoteAlgorithm(roundConfig, spec) → requires remote id/version to match the local algorithm
5. mlRuntimeSupport.requireSupported(spec)   → validates target (JVM, Android, Apple Silicon)
6. DataProfileService.profile(...)           → generates local profile
7. DefaultPreflightValidator.validate(...)   → runs checks & registers preflight hash on ledger
8. DataComponent.authorizedTrainingView(...) → creates contract-filtered view
9. AuthorizedTrainingViewWriter.writeCsv(...)→ materializes authorized_view.csv in runtimeRoot
10. resolveGlobalModel(roundConfig)          → downloads previous model for FedProx via artifactChannel
11. TrainingLocalModel.execute(context)      → runs the local algorithm
12. validateTrainingResult(...)              → TrainingStatus.SUCCESS, maxUpdateNorm, OutputPolicy files/size
13. writeDataProfile(...) + validateUpdateVector(...) → non-empty, finite, norm ≤ 1e-6 diff, pre-DP SHA-256 hash check
14. DataComponent.applyDifferentialPrivacyIfConfigured(...) → centralized DP (clip + noise from research.xml)
15. artifactChannel.upload(...)              → sends update, metrics, profile via the digital twin (or local disk for ephemeral Docker)
16. signatureService.sign(...)               → signs the update hash
17. experimentChaincode.registerLocalUpdateHash(...) & registerClientMetricsHash(...) → registers on Fabric ledger
    (BL-261: ClientUpdateAsset also carries the update vector CSV, sampleCount/positiveCount, norm, and metric)

Data sovereignty boundary — no direct Artifact Experiment Service access: FederatedClientService takes a ClientArtifactChannel (not a LocalArtifactExperimentService) for both writing round artifacts and downloading the global model for FedProx. In production/research composition roots this is TwinArtifactChannel, which publishes ClientArtifactPayload/reads GlobalModelPayload over the same digital-twin interface that already carries ClientUpdateMessage — see 17-local-digital-twin-broker.md. The Artifact Experiment Service is written to only by the orchestrator's ClientArtifactIngestor after the bytes arrive, never by the client process itself. The one documented exception is LocalDiskArtifactChannel, used only by the ephemeral, host-orchestrated Docker path (Docker*ClientApp — no live orchestrator/broker counterpart reachable from inside that short-lived container); see that class's Javadoc for the full rationale.

Result:

public record ClientExecutionResult(
    String             clientId,
    TrainingStatus     status,       // UPDATE_SUBMITTED | PREFLIGHT_FAILED | FAILURE
    PreflightResult    preflightResult,
    Optional<ClientUpdate> update,
    double             localMetric,
    String             localMetricName,
    List<EpochPoint>   trainingHistory,   // per epoch: loss, accuracy, gradNorm
    Map<String,Double> perClassAccuracy   // HAR: WALKING=0.92, SITTING=0.87, ...
)

FederatedDockerClientExecutor

Launches Docker containers, mounts volumes, and reads result.properties for each client.

FederatedDockerClientExecutor executor = new FederatedDockerClientExecutor(
    Path.of("").toAbsolutePath().normalize(), // projectRoot
    "br.ufma.lsdi.research.har.DockerHarClientApp"  // MAIN_CLASS
);

FederatedDockerClientExecutor.Result result = executor.execute(
    contract, roundConfig,
    List.of("1", "2", "3", "5", "8"),  // selected clientIds
    "hdt-af-client:local",              // Docker image
    true                                // printOutput
);

List<ClientUpdate>     updates   = result.updates();
List<ClientRunSummary> summaries = result.clientSummaries();
List<ContainerRunSummary> containers = result.containers();

Environment variables passed via -e to each container:

Variable Example
MAIN_CLASS br.ufma.lsdi.research.har.DockerHarClientApp
CLIENT_ID 3
EXPERIMENT_ID exp-har-smartphones-001
ROUND_ID 1
GLOBAL_MODEL_VERSION 0
GLOBAL_MODEL_URI file://target/artifacts/.../model.bin
GLOBAL_MODEL_HASH sha256:abc123...
SELECTED_CLIENTS 1,2,3,5,8
HDT_AF_DATA_ROOT /app/data
HDT_AF_ARTIFACT_ROOT /app/artifacts
HDT_AF_RUNTIME_ROOT /app/runtime
HDT_AF_RESEARCH_ROOT /app/research
RESULT_FILE /app/output/result.properties

Docker availability check:

boolean dockerOk = FederatedDockerClientExecutor.isDockerAvailable();
// Returns false if the Docker daemon does not respond — used in tests with assumeTrue()

Note: executeRoundEnvelopeAndPublish(...) calls roundConfigEnvelope.verify(signatureService) (without calling verifyPayloadIntegrity()), whereas executeRoundOnFinalizationSignal(...) and FederatedMqttClientOrchestrator verify both signature and payload integrity.

Volumes mounted per container (FederatedDockerClientExecutor)

Local volume Mount in container Mode
target/classes /app/classes :ro
target/lib /app/lib :ro
target/docker-client-data/{exp}/round-{r}/{clientId}/timeseries.db /app/data/timeseries.db :ro
target/artifacts /app/artifacts :rw
target/docker-runtime/{exp}/round-{r}/clients/{c} /app/runtime :rw
target/docker-sim/{exp}/round-{r}/clients/{c} /app/output :rw
logs/docker/{exp}/round-{r}/clients/{c} /app/logs :rw
research/experiments/{experimentId} /app/research/experiments/{experimentId} :ro

In addition to the experiment/path variables above, FederatedDockerClientExecutor invokes DockerMqttEnvironment.appendMqttEnvironment(command, "host.docker.internal", true) so containers receive the active MQTT_* and SIGNING_* settings.

Container logs reach the host (BL-231): docker/client/Dockerfile sets WORKDIR /app. Each client/round receives a dedicated host log directory mounted at /app/logs (note: modules/federated-common/src/main/resources/logback.xml is currently absent from the checkout, so container log routing depends on Logback configuration on the classpath). Created ahead of docker run by ensureDockerRuntimeLayout.

Training data and model exposed under logs/ too (BL-232): after each container finishes, execute() calls linkContainerArtifactsIntoLogs, which creates two symlinks (not copies) under logs/<containerName>/:

Symlink Points to Content
logs/<containerName>/data target/docker-client-data/{exp}/round-{r}/{clientId} the filtered timeseries.db actually used for training
logs/<containerName>/model target/artifacts/{experimentId}/{clientId}/round-{roundId} local_update.bin, metrics.json, data_profile.json produced this round

The model symlink is only created if that directory actually exists — a client that never produced an update (e.g. rejected at preflight) has nothing to point to, and the method skips it silently rather than creating a broken link. Symlinks, not copies: the source directories already live on the host, so there's no duplication and nothing to keep in sync.

Isolation invariant (BL-091): DataComponent.loadLocalCdmRepository() reads directly from timeseries.db (no longer from data/{clientId}/{experimentId}/dataset.csv), so mounting the shared timeseries.db from the entire data/ directory into the container would expose data from all clients. Before each docker run, FederatedDockerClientExecutor calls ClientTimeSeriesExporter to generate, from the shared timeseries.db, a filtered database containing only the rows for that container's (clientId, experimentId) — it is this filtered file, not the data/{clientId}/{experimentId} directory (which is no longer created/required), that is mounted read-only. See 07-data-component.md.


Docker entry points per experiment

Experiment MAIN_CLASS Class / Mode
HAR (Smartphones) br.ufma.lsdi.research.har.DockerHarClientApp HarTrainingLocalModel (ephemeral via FederatedDockerClientExecutor)
ECG5000 (Anomalies) br.ufma.lsdi.research.ecg.DockerEcgClientApp Ecg5000TrainingLocalModel + DP (ephemeral via FederatedDockerClientExecutor)
Ergonomic Risk br.ufma.lsdi.research.ergonomic.DockerErgonomicClientDaemonApp Persistent daemon via DockerParticipantLaunchService (DockerParticipantDaemonRegistry)
Synthetic Smoke br.ufma.lsdi.research.smoke.DockerSmokeClientDaemonApp Persistent daemon (docker-compose.yml --profile smoke)

Note: docker/client/Dockerfile still declares ENV MAIN_CLASS=br.ufma.lsdi.research.dt.DockerClientSimulationApp as a fallback default, which does not exist in Java sources — callers always pass MAIN_CLASS explicitly via -e.

Each ephemeral Docker app: 1. Reads required env vars (CLIENT_ID, EXPERIMENT_ID, ROUND_ID) 2. Rebuilds the contract with an empty allowedObservationCodes (allow all) 3. Loads the CDM directly from timeseries.db via DataComponent.loadLocalCdmRepository(clientId, EXPERIMENT_ID) (structural MQTT provenance, BL-091) 4. Creates FederatedClientService and calls executeRound() 5. Writes result.properties to /app/output/ (including dataProfileUri and dataProfileHash)

The reusable event-driven Docker/JVM composition is FederatedMqttClientOrchestrator in federated-client. It starts the MqttTimeSeriesIngestor subscriptions for that participant, retains only signed RoundConfig messages for the configured experiment/version, and schedules training only after a matching signed RoundFinalizationSignal. The Local CDM ingestor validates observation envelopes and persists them in the participant's local time-series store; the normal consent and authorized-view checks still run in FederatedClientService. The resulting signed ClientUpdateMessage is published through the injected MqttGateway.

FederatedDockerClientExecutor and the existing experiment-specific Docker*ClientApp entry points still use the host-exported, per-client SQLite snapshot and result file. They have not yet been switched to the event-driven orchestrator; that class provides the broker-driven daemon composition but does not make the current executor MQTT-first by itself.

Android client (modules/federated-android-client)

The Android application is a separate Gradle build outside the Maven reactor (:core, :app, :client-engine-plugin, plus composite includeBuild("../../../hdt-mobile-apps")). It shares the federated wire types from federated-common, while CDDL/CDAL provides local sensor acquisition. It is a distinct execution path from the JVM/Docker client, not a Docker client running on Android. The platform-independent mobile inference process (consent gate, observation model/assembly, round lifecycle, training engines and package/plugin contracts) is owned by federated-client under br.ufma.lsdi.fedandroid.core (ConsentManager, ObservationStore / TabularAssembler, MultiExperimentRoundCoordinator / FederatedClientOrchestrator, RoundLifecycle, LocalTrainer, ClientPackageVerifier, MdplTrainingPluginResolver). Unlike FederatedClientService, RoundLifecycle enforces RoundConfig.deadline() via Clock, re-checks consent after training, and verifies uploaded.updateHash(), while skipping selected_clients, DefaultPreflightValidator, DP, and OutputPolicy. The Android :core build compiles these source directories directly from the Maven module. SQLite, sensor callbacks, MQTT service composition, REST artifact transport and Android UI (MainActivity, ConsentActivity, ExperimentDetailActivity, CollectedDataActivity, ExperimentMonitorActivity) remain in federated-android-client.

Android client (modules/federated-android-client)

One foreground FederatedMqttService subscribes to the cross-experiment filters fl/experiments/+/rounds/+/config, fl/experiments/+/rounds/+/finalization, and fl/experiments/+/contracts/published. For each (experimentId, contractVersion), it maintains an independent RoundLifecycle and collection state. A round is processed only after validating the signed envelope, matching its identity to the payload, and checking local consent.

MQTT connectivity and CDDL sensor capture are shared across consented experiments. Readings are written to SQLite under both experiment ID and contract version; a round assembles data only from its matching scope. The version-aware schema migration leaves legacy rows without a contract version, so they are not silently reused for training under a newly accepted contract. Revoking a contract removes its pending round, plugin registration, and scoped observations. Other consented experiments continue; if no consent remains, sensor capture is stopped.

The app can also consume iot/clients/{clientId}/entities/+/observations. This is an input path; the Android client does not publish its local sensor readings to the remote federated broker. ClientUpdateRelay continues to relay the signed ClientUpdateMessage from the client-facing topic to the aggregator-facing topic; the client never connects directly to the aggregator.

CddlLiveObservationStore keeps at most 150 recent readings in process memory for the local preview screen. It is not training storage and is cleared when the service stops. Training observations are persisted only while a consented round is collecting.

Independent MDPL client package

An experiment may publish a separate .hdtclient.zip; it is not part of the server/experiment package and does not require a new APK installation. It contains the DEX, a descriptor binding experiment, contract version, algorithm ID/version, plugin class and DEX hash, SHA-256 checksums, and an Ed25519 signature. The private signing key stays outside the repository and APK.

Before loading the DEX, Android fetches the client package for the exact experiment/version, checks the pinned public key and key ID, verifies the signature, checksums and contract binding, then resolves the declared plugin through DexClassLoader. The resolver also checks that the plugin's algorithm ID/version match the signed descriptor and that its MlAlgorithmSpec supports Android Java client-side federated training. An absent package uses the built-in LocalTrainer; an invalid package is rejected rather than treated as a successful fallback.

The plugin algorithm runs inside FederatedMdplInferenceEngineService, whose Android-specific source is maintained in federated-client/src/android/java. It is a subclass of MDPL's OnArrivalPhenotypeInferenceEngine. Requests and results use the device-local CDDL topics mdpl/clients/{clientId}/training/requests, .../results, and .../ready; they are not published to the remote MQTT broker. The engine emits readiness only after it has subscribed to the local request topic. RoundLifecycle receives the resulting TrainingOutcome, uploads the authorized artifacts through the REST artifact API, and publishes the signed update over the normal client-facing MQTT topic.

FederatedClientOrchestrator (in federated-client) routes signed round messages to independent RoundLifecycle instances keyed by experiment and contract version. FederatedMdplOrchestrator owns preparation and release of the per-contract training engine. Its MDPL package manager, registry and CDDL request/result bridge are colocated in the Android source set of federated-client. The service conforms to MDPL's OnDataArrivalInference through its OnArrivalPhenotypeInferenceEngine base and dispatches into the shared MdplTrainingPlugin contract. The Android service still chooses and releases the engine per contract. For each (experimentId, contractVersion), it obtains the published client package, asks MdplClientPackageManager to verify and load it, registers the plugin with the MDPL runtime, and provides a MdplTrainingEngine to that contract's RoundLifecycle. The package hash is pinned to the contract version; replacing a package requires a new version. If the server returns 404 because no client package exists for the contract, the orchestrator selects the built-in LocalTrainer. Invalid or mismatched packages fail round preparation and do not silently fall back. Consent revocation releases that contract's engine and plugin registration; stopping the last consented capture releases the remaining in-memory registrations.

The MDPL OnDataArrivalInference interface has an optional federated-training capability: supportsFederatedTraining() and executeFederatedTraining(FederatedTrainingRequest). Existing inference engines remain compatible through default methods that report no FL support. The request binds the local payload to client, experiment, contract version, round and algorithm; the result returns the model update and aggregate metrics. The Android MDPL service implements this contract and dispatches only to the package registered for the exact experiment/version. The payload moves through local CDDL request/result topics and remains on device; RoundLifecycle is still responsible for consent checks, collecting the authorized view and publishing the signed FL update.

The device's MDPL request/result channel is the local CDDL microbroker. The federated update is published by the Android MQTT transport to the regular fl/.../updates topic after the MDPL result returns. The Android composite build resolves cdal.aar from the sibling hdt-mobile-apps checkout. Inspection of that AAR found sensor/data-collection classes but no MQTT Publisher or federated response API; the publisher API in use belongs to CDDL, and the remote FL response is sent by Paho. CDDL/CDAL sensor capture remains local, and raw observations are not sent to the remote broker by the Android client.

Package creation and optional independent deployment use ../../scripts/package-federated-client.sh. The server trust key is configured with HDT_AF_CLIENT_PACKAGE_TRUSTED_PUBLIC_KEY and HDT_AF_CLIENT_PACKAGE_TRUSTED_KEY_ID; the APK pins the corresponding public key using CLIENT_PACKAGE_TRUSTED_PUBLIC_KEY and CLIENT_PACKAGE_TRUSTED_KEY_ID at Gradle build time.

On startup and when the user refreshes the catalog, the app queries GET {experiment_discovery_url}/api/experiments, validates active experiment IDs and contract versions, and caches the catalog and per-contract configuration in encrypted preferences. contractPublished is received with a cross-experiment wildcard; the HMAC and payload integrity are checked before refreshing the catalog. Discovery never grants consent or starts training. The user must explicitly accept each (experimentId, contractVersion) pair.

The MQTT round protocol remains MQTT-only; REST is used for discovery, contract/code metadata, client-package download, consent synchronization, and round artifacts. The Android RestSseGateway and round-protocol REST+SSE endpoints are not used. See the topic map for the remote filters and local CDDL topics.

The client module is outside the Maven reactor. Its focused validation includes ./gradlew :core:test, ./gradlew :app:testDebugUnitTest, ./gradlew :app:compileDebugJavaWithJavac, and the MDPL/plugin tasks.

ExperimentDetailActivity forwards the exact (experimentId, contractVersion) pair to ConsentActivity. Accepting, declining, and revoking affect only that contract. Revocation deletes the scoped SQLite observations and notifies the running service; the service stops that experiment's collection and unloads its plugin without stopping other consented experiments.

Anonymous installation identity (BL-103)

On first use, ClientConfig creates a random UUID v4 exactly once with UUID.randomUUID() and keeps it in EncryptedSharedPreferences. The value is not derived from IMEI, Android ID, MAC address, account, location, or sensors. The service registers only this UUID and the app version after at least one contract has consent via POST {experiment_discovery_url}/api/client-installations; the submission does not include clientId, does not go to MQTT/Fabric, and carries no raw data. The API is idempotent: 201 on the first registration and 200 on subsequent contacts.

When accepting or declining the terms, Android first keeps the local decision and sends the decision to the monitor in the background. RestConsentSyncClient uses the installation's anonymous UUID as the client identifier and executes action=add followed by action=accept or action=reject on POST /api/experiments/{experimentId}/clients/{anonymousUuid}. As a result, GET /api/experiments/{id} starts including the client in the consents array, which the web application's Consents tab displays.

BL-109: the details screen queries GET /api/experiments/{id}?version={version} when opening and when returning from the decision. It identifies the current anonymous UUID in the consents array and shows only Accept terms with no remote acceptance, or only Revoke when the remote status is ACCEPTED (with compatibility for EXPCEPTED). The terms screen waits for the REST POST to complete before returning, eliminating the race between acceptance and reading the remote state. Network failures do not remove the local acceptance nor release collection or training without the local gate.


result.properties — container → host communication

experimentId=exp-har-smartphones-001
contractVersion=1.0.0
clientId=3
status=UPDATE_SUBMITTED
sampleCount=350
positiveCount=0
updateNorm=312.45
localMetric=0.8312
localMetricName=train_accuracy
failedChecks=
roundId=1
updateUri=file://target/artifacts/exp-har-smartphones-001/3/round-1/local_update.bin
updateHash=sha256:3a7bd3e2...
metricsUri=file://target/artifacts/...
dataProfileUri=file://target/artifacts/exp-har-smartphones-001/3/round-1/data_profile.json
dataProfileHash=sha256:9c12b4f0...
updateVectorCsv=0.012,−0.034,...  (full CSV vector)
signature=hmac:client-3:abc123...

Preflight — checks run before training (DefaultPreflightValidator)

Check Condition
minimum_examples_per_client sampleCount ≥ minimumExamplesPerClient
minimum_positive_examples_per_client positiveCount ≥ minimumPositiveExamplesPerClient
minimum_entities_per_client entityCount ≥ minimumEntitiesPerClient
expected_positive_rate_range positive rate within [minExpectedPositiveRate, maxExpectedPositiveRate]
missingness_threshold:{code} missingness rate for required property code ≤ threshold

Failure of any check → status=PREFLIGHT_FAILED, failedChecks=[check1,check2].


Update vector validation

After TrainingLocalModel.execute(), FederatedClientService validates:

  1. Vector is not empty
  2. All values are finite (no NaN, Inf)
  3. Calculated L2 norm vs. declared norm — tolerance 1e-6
  4. Recalculated SHA-256 hash of local_update.bin vs. result.updateHash()
  5. sampleCount within the authorized profile

Any mismatch → ErrorCode.UPDATE_REJECTED.


Mandatory rules

  • Do not run training without valid acceptance (ConsentStatus.isAccepted()).
  • Validate TrainingLocalModel.algorithmSpec() with ClientMlRuntimeSupport before accessing data.
  • Create the authorized view (AuthorizedTrainingView) before calling the algorithm.
  • Run the statistical preflight before training.
  • Do not send raw data to the orchestrator, aggregator, or MQTT.
  • The client's dataset must originate from timeseries.db, whose only production writer is MqttTimeSeriesIngestor — MQTT provenance is guaranteed structurally (BL-091), not by a runtime requireMqttProvenance()/source=mqtt check (that check was removed; see 07-data-component.md).
  • Never publish ClientUpdateMessage anywhere but the client-facing topic (BL-242) — the aggregator-facing topic is reserved for the orchestrator's ClientUpdateRelay.
  • Never access the Artifact Experiment Service directly (neither to write the local update/ metrics/profile nor to read the global model) — always through ClientArtifactChannel (TwinArtifactChannel in production/research; LocalDiskArtifactChannel only for the documented ephemeral Docker exception).

Docker environment variables

Variable Default Description
TIMESERIES_STORE sqlite Time-series store in the container
SIM_HMAC_SECRET web-sim-secret-dev HMAC secret for simulation sessions

Tests

Covered by ArchitectureComponentTestSuite: - Contract acceptance, preflight, training, hash/norm validation, update publication - FederatedDockerClientExecutor.isDockerAvailable() — guard for e2e tests