Component: Experiment SDK¶
Responsibility¶
The Experiment SDK defines the Java contract that researchers must follow to implement local training algorithms. It separates the federated framework from the scientific code specific to each experiment, ensuring that no framework dependency leaks into the research code.
Implementation¶
| Item | Value |
|---|---|
| Module | modules/federated-experiment-sdk |
| Java package | br.ufma.lsdi.experiment.sdk |
| Main interfaces & exceptions | TrainingLocalModel, TrainingContext, TrainingResult, TrainingException, ExperimentSession, ExperimentSessionFactory |
| ML catalog | MlAlgorithmSpec, MlRuntimeTarget, MlAlgorithmCatalog |
| Utilities | ExperimentRuntimePaths, ConsentTermsMarkdownReader, RoundEvaluationReport (EpochPoint lives in br.ufma.lsdi.common.model inside federated-common) |
Main interface: TrainingLocalModel¶
This is the only interface the researcher needs to implement. The framework does not require any dependency beyond the SDK (and federated-common) to integrate a new experiment.
public interface TrainingLocalModel {
/** Declares the algorithm and its supported targets. Validated before data access. */
default MlAlgorithmSpec algorithmSpec() {
return MlAlgorithmCatalog.CUSTOM_JAVA_TRAINING_JOB;
}
/**
* Executes local training.
*
* @param context round metadata, authorized data view path, outputPath, hyperparameters
*/
TrainingResult execute(TrainingContext context) throws TrainingException;
}
Mandatory rules for TrainingLocalModel:
- Never open
data/{client_id}/{experiment_id}/dataset.csvdirectly — read fromcontext.authorizedDataView()(Pathtoauthorized_view.csv). - Never write files outside
context.outputPath(). - Never connect to the network or external services.
- Never return an empty vector or one containing
NaN/Inf. - Declare
algorithmSpec()with supported targets. - Produce per-epoch history (
trainingHistory) and per-class accuracy (perClassAccuracy) when available — the web monitor displays them per client.
TrainingContext¶
Groups all execution context for a round without exposing the raw dataset.
| Field | Type | Description |
|---|---|---|
experimentId() |
String |
Experiment identifier |
contractVersion() |
String |
Contract version |
roundId() |
long |
Current round number |
authorizedDataView() |
Path |
Path to the CSV generated by AuthorizedTrainingViewWriter — the only lawful data access |
outputPath() |
Path |
Directory where local_update.bin, metrics.json, and data_profile.json are written |
globalModelPath() |
Path |
Path to previous global model weights (nullable, used by FedProx/FedAdam) |
hyperparameters() |
Map<String,String> |
Hyperparameters published in RoundConfig (epochs, lr, mu, dropout, …) |
privacyPolicy() |
PrivacyPolicy |
Privacy policy metadata from RoundConfig |
studyDesign() |
StudyDesign |
Study design metadata from the contract |
aggregationPolicy() |
AggregationPolicy |
Aggregation policy metadata |
dataWindow() |
TimeWindow |
Optional temporal window for the round |
TrainingResult¶
Represents all artifacts and metrics produced by local training (also offers an 8-arg compatibility constructor without localMetric/localMetricName/trainingHistory/perClassAccuracy).
| Field | Type | Description |
|---|---|---|
localUpdatePath |
Path |
Path to local_update.bin (CSV of doubles) |
metricsPath |
Path |
Path to metrics.json |
dataProfilePath |
Path |
Path to data_profile.json |
updateHash |
String |
SHA-256 of the contents of local_update.bin |
updateNorm |
double |
L2 norm of the update vector |
status |
TrainingStatus |
SUCCESS, FAILED, PREFLIGHT_FAILED, COMPLETED_NOT_SUBMITTED, UPDATE_SUBMITTED |
sampleCount |
int |
Total number of samples used |
positiveCount |
int |
Positive-class samples (binary classification) |
localMetric |
double |
Value of the training metric (e.g., accuracy) |
localMetricName |
String |
Metric name (e.g., "train_accuracy") |
trainingHistory |
List<EpochPoint> |
Convergence history per epoch |
perClassAccuracy |
Map<String,Double> |
Accuracy per class label (displayed as bars in the monitor) |
EpochPoint (federated-common)¶
Record of one training epoch, propagated to the web monitor.
The monitor displays a table of epochs and the per-class accuracy chart for each client on the round detail page. Essential for detecting client drift.
MlAlgorithmSpec¶
Declares the ML algorithm and the targets supported by the TrainingLocalModel.
public record MlAlgorithmSpec(
String algorithmId, // catalog ID (e.g., "federated_mlp_classifier")
String algorithmFamily, // family (e.g., "multiclass_classification")
String updateFormat, // update format (e.g., "neural_network_weights_csv")
boolean trainableOnClient, // whether it can be trained locally
boolean supportsFederatedUpdate,// whether it produces a federatable update
List<MlRuntimeTarget> supportedTargets // compatible targets
)
MlRuntimeTarget¶
Enum with the supported execution targets:
| Value | Description |
|---|---|
JVM_SERVER |
Standard JVM — server or desktop |
EDGE_JVM |
Embedded JVM (limited resources) |
ANDROID_JAVA |
Android with plain Java (lightweight algorithms) |
ANDROID_LITERT |
Android LiteRT/TensorFlow Lite (trainable models) |
ANDROID_ONNX |
Android ONNX Runtime (inference/validation) |
MACOS_APPLE_SILICON_JAVA |
macOS Apple Silicon with AArch64 JDK |
MACOS_APPLE_SILICON_DJL |
macOS Apple Silicon with DJL for execution/validation |
The client validates ClientMlRuntimeSupport against the declared targets before granting access to the dataset.
MlAlgorithmCatalog¶
Static catalog of the framework's native algorithms:
| ID | Description |
|---|---|
federated_mlp_classifier |
MLP for multiclass classification (HAR experiment) |
federated_mlp_autoencoder |
MLP Autoencoder for anomaly detection (ECG experiment) |
federated_centroid_classifier |
Centroid-based classifier for resource-constrained environments |
federated_dt_regressor |
Regression for industrial digital twins (DT experiment) |
Researchers can register algorithms outside the catalog by providing a custom algorithmId in MlAlgorithmSpec.
ExperimentSession and ExperimentSessionFactory (ADR-005)¶
Moved from br.ufma.lsdi.admin.AdminExperimentSession to br.ufma.lsdi.experiment.sdk so that
packaged experiments (see docs/components/15-experiment-packaging.md) can
implement them without depending on the entire federated-rest-api module.
public interface ExperimentSession {
String experimentId();
String contractVersion();
FederatedExperimentRunResult executeRound(long roundId, String phase);
}
public interface ExperimentSessionFactory {
ExperimentSession create(Path datasetRoot, String contractVersion, int maxClients);
}
ExperimentSessionkeeps infrastructure alive between rounds (loaded dataset, global weights, coordinator, aggregator) — each implementation (HAR, ECG, or a dynamically deployed experiment) initializes once and exposesexecuteRoundfor incremental execution.ExperimentSessionFactoryis the extension point referenced by thesessionFactoryClassfield of a package'spackage-descriptor.json(ExperimentPackage) — validated and, for every round, actually loaded/executed inside a genuinely separate isolated OS process (own PID/JVM), never inside admin-api's own JVM (ExperimentDeploymentService/br.ufma.lsdi.admin.isolated.*,federated-rest-apimodule, BL-236).datasetRootmay benull: the implementation resolves a default path (e.g.,EcgSessionFactorydelegates toEcg5000DatasetLoader, which falls back toExperimentRuntimePaths.datasetCacheRoot(...)).
ExperimentRuntimePaths¶
Resolves all system paths in a way that is portable between local execution and Docker.
| Method | Return | Environment variable | Default |
|---|---|---|---|
dataRoot() |
Path |
HDT_AF_DATA_ROOT |
data |
artifactRoot() |
Path |
HDT_AF_ARTIFACT_ROOT |
target/artifacts |
runtimeRoot() |
Path |
HDT_AF_RUNTIME_ROOT |
target/runtime |
researchRoot() |
Path |
HDT_AF_RESEARCH_ROOT |
research |
deployedExperimentsRoot() |
Path |
HDT_AF_DEPLOYED_EXPERIMENTS_ROOT / DEPLOYED_EXPERIMENTS_ROOT |
target/deployed-experiments |
deployedExperimentDirectory(experimentId, contractVersion, shortHash) resolves a directory
isolated per deploy within deployedExperimentsRoot() — this prevents a new upload from overwriting a
jar already in use by an isolated process spawned for a running session (see ADR-005, BL-236).
All experiments must use ExperimentRuntimePaths instead of literal strings.
Mandatory rules¶
- Every
TrainingLocalModelmust declarealgorithmSpec(). - An algorithm with no target compatible with the client's environment must be blocked before data access.
- An algorithm marked
trainableOnClient=falsemust not be accepted for federated training. - Researcher code must live in
research/experiments/{experiment_id}. - Dynamic deploy (ADR-005):
ExperimentDeploymentServiceloads the package jar without verifying a signature, and the/experiments/deployendpoint still has no access control of its own — a risk explicitly accepted by the operator of this MVP. Since BL-236, the loaded code always runs in a genuinely separate OS process, never inside admin-api's own JVM — see "Process isolation (BL-236)" and "Accepted risk" indocs/components/15-experiment-packaging.md. Jar signing and resource limits on the isolated process remain future hardening items, not implemented. localUpdatePathmust reference a file written underoutputDir, not an arbitrary path.updateHashmust be the SHA-256 of the contents oflocal_update.bin; a mismatch with the hash recalculated by the client is grounds for rejection (ErrorCode.UPDATE_REJECTED).
Current state¶
The SDK supports JVM, Android Java, Android LiteRT, Android ONNX Runtime, macOS Apple Silicon Java, and macOS Apple Silicon DJL targets. The exp-har-smartphones-001 and exp-ecg5000-anomaly-001 experiments produce trainingHistory with EpochPoint and perClassAccuracy, visible in the web monitor per client and round.
Tests¶
Covered by federated-system-tests:
FederatedSystemTests— validates theTrainingLocalModelcontract across all experiments end to endArchitectureComponentTestSuite— experiment SDK, TrainingLocalModel, TrainingResult, EpochPoint, supported algorithms
RoundEvaluationReport — evaluating a multi-round run¶
Renders a run of any ExperimentSession — per-client metrics per round, global-model evolution,
and an explicit list of findings about the run's scientific quality. Pure logic, no I/O.
List<FederatedExperimentRunResult> rounds = ...; // one per executeRound(...)
System.out.print(RoundEvaluationReport.render(rounds, requestedClients));
List<RoundEvaluationReport.Finding> findings =
RoundEvaluationReport.analyse(rounds, requestedClients);
Why the findings exist¶
A MIMIC run of 5 rounds finished with every round in quorum, every client UPDATE_SUBMITTED, and
train_accuracy of 1.000 — and still supported no conclusion: the metric was saturated, there was
no holdout, and the global model's drift grew between rounds instead of shrinking. Every one of
those observations required reading the numbers by hand.
A green pipeline is not a learning result. The report separates the two: what ran, and what the numbers permit you to claim.
Findings¶
| Code | Severity | Detects |
|---|---|---|
no-rounds |
ERROR | nothing was executed |
quorum-not-satisfied |
ERROR | a round aggregated without the minimum clients |
client-count-reduced |
WARNING | fewer effective clients than requested — the dataset cannot support the requested federation, and the truncation is silent |
not-converging |
WARNING | drift between successive global models ends higher than it started |
metric-saturated |
WARNING | the local metric is identical across every client and round — no discriminating power |
no-holdout-metric |
WARNING | globalEvaluationMetric is NaN in every round — no central evaluation |
update-exceeds-model |
WARNING | mean ‖update‖ exceeds ‖global model‖ — each round rewrites rather than refines |
no-promotion-decision |
INFO | no round recorded PROMOTED/REJECTED |
Convergence is judged by drift — L2 distance between successive global models. A converging run produces models that get closer together; if the final drift exceeds the first, the model is wandering. Needs at least 3 rounds to have an opinion.
Runners¶
Per-experiment main() apps render the report. MIMIC:
sh scripts/run-mimic-experiment.sh # synthetic fallback, 4 clients, 5 rounds
MIMIC_NUM_ROUNDS=10 sh scripts/run-mimic-experiment.sh /path/to/dataset
| Variable | Default | Description |
|---|---|---|
MIMIC_MAX_CLIENTS |
4 |
clients per round (the dataset supports at most 4) |
MIMIC_NUM_ROUNDS |
5 |
federated rounds |
MIMIC_DATASET_ROOT |
— | dataset root; absent uses the synthetic fallback |
MIMIC_USE_BROKER |
false |
true connects to a real MQTT broker via Paho |
The app exits 1 when any ERROR-severity finding is present, so it can be chained in CI without
parsing the report text.
Without a real
cohort.csv, the run uses the synthetic fallback (10 admissions per client). That exercises the pipeline and says nothing about ICU mortality — the report flags it.