Model inference service¶
federated-model-inference stores complete trained model packages as immutable Fabric world-state assets, serves them over HTTP, and evaluates records supplied over HTTP. It uses the same peer TLS certificate and application MSP identity variables as the rest of the project.
Current training and artifact flow¶
Each research experiment implements a Java TrainingLocalModel/session and trains a domain-specific model. The federated client publishes numeric updates; aggregation stores the resulting global weight vector under the experiment artifact root (commonly global_model.bin, with CSV-encoded doubles). The algorithm choice and hyperparameters are recorded in experiment configuration and aggregation reports. Existing .hdtpkg.zip files contain experiment code and a session factory, not the fitted model weights. A deployable inference bundle therefore needs both the fitted artifact and explicit runtime metadata.
Inference package format (.hdtmodel.zip)¶
Each package contains:
model-manifest.json: format version, model identity/version, algorithm ID, artifact filename, ordered input feature names, algorithm parameters, training hyperparameters, and output description.- The fitted model artifact (for the included linear adapters this is a CSV/whitespace vector of feature weights followed by bias).
SHA256SUMS, written by the shared package ZIP utility.
ModelPackageWriter creates packages. ModelPackageReader extracts with the existing zip-slip and entry-size protections and requires valid checksums before loading the artifact. A package is limited to 2 MiB compressed to fit the Fabric Gateway transaction envelope. The chaincode stores the complete ZIP as Base64 together with its SHA-256 and manifest metadata. Re-registering the same (modelId, modelVersion) is rejected; package updates use a new version.
Algorithms implement InferenceAlgorithm. ModelInferenceService supports ServiceLoader in its 2-arg constructor, while ModelInferenceApplication.main currently calls the 3-arg constructor passing a fixed list (linear-regression-v1, logistic-regression-v1), ignoring extra ServiceLoader providers unless wired explicitly. inferLatest compares modelVersion values lexicographically (String::compareTo), so "1.10.0" sorts before "1.2.0". Other experiment architectures (MLP, autoencoder, etc.) require a provider that understands their architecture and artifact serialization; an update vector alone is not enough to reconstruct inference behavior.
HTTP API¶
Start the service with sh scripts/run-model-inference.sh. The script loads .env (and deploy/fabric/fabric-env.sh when available), checks the Fabric TLS/MSP files, compiles the project, and starts ModelInferenceApplication. Configure these optional service variables (ModelInferenceApplication reads System.getenv only, unlike FabricInferencePackageLedger which accepts -D properties too):
| Variable | Default | Purpose |
|---|---|---|
HDT_MODEL_WORK_DIR |
target/model-work |
Verified extraction cache |
HDT_MODEL_API_HOST |
127.0.0.1 |
Bind address |
HDT_MODEL_API_PORT |
8092 |
HTTP port |
The Fabric connection uses FABRIC_GATEWAY_PEER_ENDPOINT, FABRIC_GATEWAY_TLS_SERVER_NAME_OVERRIDE, FABRIC_GATEWAY_TLS_CERT_PATH, FABRIC_GATEWAY_MSP_ID, FABRIC_GATEWAY_CERT_PATH, FABRIC_GATEWAY_KEY_PATH, FABRIC_CHANNEL_NAME, and FABRIC_CHAINCODE_NAME.
After adding the new Fabric transactions, build and deploy an updated federated-governance chaincode package before starting the API against the network:
(cd deploy/fabric/chaincode/federated-governance-java && gradle shadowJar)
sh scripts/fabric-package-chaincode.sh
Then install/approve/commit the generated package using the network's normal Fabric lifecycle. The API fails on Fabric read/write errors and does not switch to local package storage.
Routes:
GET /modelslists on-chain package IDs, versions, and SHA-256 values.GET /models/{modelId}/package?version={version}downloads the complete package ZIP from Fabric.POST /modelsstores a complete.hdtmodel.ziprequest body on Fabric after validating the manifest and checksums; response201returns its ID, version, and hash.
Upload and retrieve a package:
curl -X POST http://127.0.0.1:8092/models \
-H 'Content-Type: application/zip' \
--data-binary @model.hdtmodel.zip
curl http://127.0.0.1:8092/models
curl -o downloaded.hdtmodel.zip \
'http://127.0.0.1:8092/models/risk-model/package?version=1.0.0'
POST /infer/{modelId} evaluates the latest package version for one model.
* POST /infer evaluates the same input records against multiple package IDs (modelIds), using each model's latest version.
* POST /infer/{modelId}/stream accepts newline-delimited JSON (NDJSON) and emits one NDJSON prediction per input row as it arrives (each line up to 1 MiB; per-row errors emit {"error":...}\n on the already-committed HTTP 200 stream).
* POST /infer/stream?modelIds={id1},{id2} performs the same streaming inference against 1..50 packages (query must start with modelIds=).
Example body:
For multi-package inference:
Package uploads are limited to 2 MiB. Inference batch and NDJSON requests are bounded to 10 MiB and 10,000 records; streaming requests emit each result immediately. Input rows are not persisted. Deploy behind the platform's TLS/authentication gateway when exposed beyond loopback. Fabric's ledger is immutable and replicated to channel members, so package access follows the channel's membership and confidentiality policy.