https://doi.or g/10.31449/inf.v48i1 1.6186 Informatica 48 (2024) 167–180 167
Efficient V anilla Split Learning for Privacy-Pr eserving Collaboration in
Resour ce-Constrained Cyber -Physical Systems
Nabila Azeri
1
, Ouided Hioual
2
and Ouassila Hioual
2, 3
1
ICOSI Laboratory , Abbes Laghrour University , 40004, Khenchela, Algeria
2
Mathematics and Informatics Department, Abbes Laghrour University , 40004, Khenchela, Algeria
3
LIRE Laboratory , Constantine 2 University , BP67A, Ali Mendjeli, 25000, Constantine, Algeria
E-mail: azeri.nabila@univ-khenchela.dz, hioual.ouided@univ-khenchela.dz, hioual_ouassila@univ-khenchela.dz
Keywords: cyber -physical systems, vanilla split machine learning, privacy-preserving learning, resource-constrained
learning
Received: May 13, 2024
In the r ealm of Cyber -Physical Systems (CPS), the integration of Federated Machine Learning (FML) al-
gorithms has become indispensable for enhancing adaptability , data privacy , and security . However , FML
falls short when deployed in r esour ce-constrained CPS envir onments due to its inher ent demands on client
r esour ces. T o addr ess this limitation, this paper intr oduces a novel Split Machine Learning (SML) ar chitec-
tur e tailor ed specifically for r esour ce-constrained CPS deployments. Unlike FML, SML strategically splits
the model between devices and a central server , enabling collaborative learning while pr eserving data pri-
vacy . Adopting a distributed learning paradigm, SML facilitates r eal-time system adaptation based on local
sensor data, mitigating communication over head and ensuring privacy . Experimental evaluation demon-
strates that the pr oposed SML-based ar chitectur e achieves an accuracy rate of 97.56% with a pr ocessing
time r eduction of appr oximately 41% compar ed to FML methods. These r esults highlight the potential of
our appr oach to impr ove collaboration in r esour ce-constrained envir onments while maintaining high lev-
els of privacy and performance.
Povzetek: Razvita je Split Machine Learning (SML) ar hitektura za izboljšanje zasebnosti in zmogljivosti
sodelovanja v kibernetsko-fizičnih sistemih (CPS) z omejenimi viri.
1 Intr oduction
In the era of interconnected technologies, Cyber -Physical
Systems (CPS) play a pivotal role in various applications,
from smart infrastructures to autonomous systems [ 14 ].
However , the dynamic nature of the real world and evolv-
ing user requirements necessitate continuous adaptation for
CPS to remain ef fective.
Machine Learning (ML) has emer ged as a powerful tool
for enhancing adaptability in CPS [ 18 ], [ 17 ], [ 16 ]. By ana-
lyzing sensor data and learning patterns, ML algorithms can
guide system adjustments and optimize performance. How-
ever , conventional ML approaches often centralize data on
a server , raising serious concerns regarding data privacy
and security . This becomes particularly critical when deal-
ing with sensitive data collected from personal devices or
industrial processes.
T o address these challenges, collaborative learning ap-
proaches such as Federated Machine Learning (FML) has
been introduced [ 21 ], [ 4 ], [ 8 ]. This technique distributes the
training process across devices, allowing models to learn
from local data without directly sharing it. While FML
of fers advantages in data privacy , its limitations become
evident in resource-constrained environments. Primarily ,
these limitations arise due to FML ’ s significant demands
on client resources. In such settings, where computational
power and bandwidth are often limited, FML struggles to
operate ef ficiently . The decentralized nature of FML re-
quires participating devices to possess suf ficient computa-
tional capabilities to handle model training tasks locally .
However , in resource-constrained environments, such as
those found in many CPS applications, devices may lack
the necessary processing power or memory to execute these
tasks ef fectively .
This paper introduces the concept of SML (Split Machine
Learning) as a particularly advantageous approach, espe-
cially for resource-constrained CPS. SML further refines
the collaborative learning concept by strategically split-
ting the model itself between devices and a central server .
By doing so, it aims to address the limitations of cen-
tralized learning approaches and the resource demands of
FML, thereby enhancing adaptability and data privacy in
resource-constrained CPS environments. Specifically , this
paper of fers the following contributions::
– Integration of SML into our previous work on self-
adaptive CPS architectures, enhancing adaptability
while preserving data privacy and security , particu-
larly in resource-constrained CPS environments.
– Application of our approach to a real-world case study

168 Informatica 48 (2024) 167–180 N. Azeri et al.
in fault prediction within an industrial system, demon-
strating its ef fectiveness in practical CPS deploy-
ments.
– Evaluation of the proposed SML-based approach
through experimental analysis reveals promising out-
comes, particularly in resource-constrained environ-
ments, with notable achievements in terms of accuracy
and performance.
The rest of the paper is structured as follows. Section
2 discusses related work in the field. Section 3 presents
an overview of our previous research ef forts, particularly
focusing on our earlier investigations and architectural pro-
posals. Section 4 elaborates on the intricacies of the pro-
posed SML-based architecture, elucidating its key compo-
nents and operational dynamics. The practical application
of our architecture on a real-world industrial system are de-
tailed in Section 5 . Section 6 of fers concluding remarks,
summarizing the findings and outlining some future work.
2 Related work
ML techniques integrated into CPS have been instrumental
in addressing challenges like self-adaptation, security , and
data privacy , enabling CPS to autonomously adapt to dy-
namic environments, enhance security measures, and safe-
guard sensitive data [ 9 , 13 ]. This section surveys existing
research on utilizing ML within CPS environments, high-
lighting its benefits in facilitating self-adaptation, strength-
ening security mechanisms, and ensuring data privacy , as
demonstrated in T able 1 . The table categorizes these works
based on several key criteria, including the specific ML
method employed, the learning process (centralized vs. dis-
tributed), and considerations for privacy , security , commu-
nication overhead, and resource constraints in CPS deploy-
ments. Following this overview , we transition to a subsec-
tion ”Discussion,” where we critically analyze the limita-
tions of existing approaches and set the stage for introduc-
ing our novel contribution.
In [ 4 ], authors propose an innovative approach for
enhancing self-adaptive CPS using FML. While many
self-adaptation techniques in CPS emphasize performance
gains, this work prioritizes security , data privacy , and
adaptability . The proposed approach leverages FML tech-
nology to reconcile adaptability with data security and pri-
vacy , addressing the challenges posed by dynamic environ-
ments, shifting user requirements, and device dynamics.
Authors in [ 10 ] propose DeepFed, a federated deep learn-
ing scheme for detecting cyber threats in industrial CPS.
The scheme utilizes a convolutional neural network (CNN)
and a gated recurrent unit (GRU) to develop an intrusion
detection model tailored for industrial CPS. Furthermore, a
federated learning framework is introduced, allowing mul-
tiple industrial CPS to collaboratively build an intrusion
detection model while preserving data privacy . A Paillier
cryptosystem-based secure communication protocol is also
devised to ensure the security and privacy of model param-
eters during the training process. Experimental results on
real industrial CPS datasets demonstrate the ef fectiveness
of DeepFed in detecting various cyber threats and its supe-
riority over existing schemes.
In [ 1 ], authors discuss the integration of ML techniques
within CPS, focusing on Industry 4.0. CPS combines com-
putation and physical processes and ML optimizes CPS
functionalities such as domain adaptation, system fine-
tuning, and vulnerability detection. ML enables CPS to
learn from lar ge-scale data, enhancing security and privacy
in industrial settings. The paper highlights ML applications
in predictive maintenance, quality assurance, and optimiza-
tion of manufacturing processes and supply chains, empha-
sizing ML ’ s transformative impact on CPS in Industry 4.0.
In their paper , W ickramasinghe et al. [ 20 ] propose a
methodology for explainable unsupervised ML tailored for
CPS. They address the challenge of the ’black-box’ nature
of complex ML models by introducing explainable unsu-
pervised ML models, particularly suitable for safety-critical
CPS applications. Their approach utilizes Self-Or ganizing
Maps (SOMs) based clustering methodology to generate
both global and local explanations, enhancing the inter -
pretability of ML models within CPS contexts. Through
feature perturbation techniques, they evaluate the fidelity
of the generated explanations, demonstrating the ef fective-
ness of their proposed method in identifying the most im-
portant features responsible for decision-making processes
in CPS.
In [ 12 ], M. Rouzbahani et al. delve into anomaly detec-
tion in CPS using ML methods. The authors underscore
the complexity of CPS, which integrate cyber components
into the physical world, leading to diverse tasks and close
interactions. W ith the proliferation of smart features and
communication tools in CPS, new challenges related to se-
curity and privacy have arisen, particularly in systems like
the smart grid. Anomaly detection emer ges as a crucial
strategy for enhancing CPS security , yet comparing various
detection methods poses challenges due to their diversity .
T o address this, the chapter focuses on ML-based anomaly
detection methods, highlighting their ef fectiveness through
a case study on False Data Injection (FDI) attacks. Through
their exploration, the authors contribute insights into the ap-
plication of ML techniques for anomaly detection in CPS,
emphasizing their potential in mitigating security threats.
In [ 15 ], authors propose a novel approach for detecting
faults in vehicular cyber -physical systems (VCPSs). They
highlight the potential of VCPSs to enhance transportation
safety , mobility , and sustainability through wireless com-
munication. However , they also address the vulnerability
of cloud-oriented architectures to cyber attacks, emphasiz-
ing the risks to safety , privacy , and property . Their pro-
posed solution involves a neural network-based technique
aimed at identifying and tracking fault data injection attacks
in real time within a platoon of connected vehicles. They
develop a decision support system to mitigate the probabil-
ity and severity of potential accidents resulting from these

Ef ficient V anilla Split Learning for Privacy-Preserving … Informatica 48 (2024) 167–180 169
T able 1: Comparative overview of ML applications in CPS
Papers Goal Method ML Learning Pr o-
cess
Privacy
Considera-
tions
Security
Considera-
tions
Commu.
Over -
head
Resour ce-
Constrained
Considera-
tions
Azeri et al.
[ 4 ]
Enhancing Self-
Adaptive CPS
using FML
Federated
Learning
Distributed Y es Y es Low No
Li et al. [ 10 ] FML for Intrusion
Detection in Indus-
trial CPS
Deep Learning
(CNN and
GRU)
Distributed Y es Y es Low No
Ahmed et al.
[ 1 ]
Machine learning in
CPS in Industry 4.0
V arious ML
techniques
Centralized Y es Y es High No
W ickram et
al. [ 20 ]
Explainable unsu-
pervised ML for
CPS
Self-
Or ganizing
Maps
Centralized No No High No
Rouzbahani
et al. [ 12 ]
Anomaly detection
in CPS using ML
V arious ML
techniques
Centralized No Y es High No
Sar golzaei
et al. [ 15 ]
Fault detection in
VCPSs using neu-
ral network-based
technique
Neural Net-
work
Centralized Y es Y es High No
Alshboul et
al. [ 2 ]
Predictive mainte-
nance in concrete
manufacturing
using ML
Feature selec-
tion
Centralized No Y es High No
Zhang et al.
[ 23 ]
Federated
Learning-based
Edge Computing
Platform for CPS
Federated
Learning
Distributed Y es Y es Low No
Xu et
al. [ 22 ]
Multiagent feder -
ated reinforcement
learning for se-
cure incentive
mechanism in CPS
Federated
Learning
Distributed Y es Y es Low No
Guo et
al. [ 7 ]
Deep federated
learning for secure
POI microservices
in CPS
Deep federated
learning
Distributed Y es Y es Low No
Our ap-
pr oach
Collaboration
Learning
in r esour ce-
constrained CPS
Split Learning Distributed Y es Y es Low Y es

170 Informatica 48 (2024) 167–180 N. Azeri et al.
attacks, ultimately enhancing system reliability , robustness,
and safety .
In [ 2 ], authors propose an empirical exploration of pre-
dictive maintenance in concrete manufacturing. By har -
nessing machine learning techniques, they aim to enhance
equipment reliability in construction project management.
The study identifies key features, such as 24-hour mean
voltage, crucial for predicting machinery failure within
the concrete manufacturing framework. Insights gleaned
from this empirical investigation underscore the signifi-
cance of integrating machine learning methodologies into
construction project management practices. This integra-
tion not only improves the accuracy of maintenance fore-
casts but also reinforces equipment dependability , ensuring
optimal ef ficacy and benefit in the concrete manufacturing
paradigm.
Authors in [ 23 ] propose a novel FML-based Edge Com-
puting platform named ”FengHuoLun” specifically de-
signed for CPS. This platform aims to address the challenge
of ensuring trustworthy smart services in Edge Comput-
ing environments by leveraging Federated Learning. W ith
FengHuoLun, smart services can be implemented with ma-
chine learning models trained in a trusted FML framework,
ensuring the trustworthiness of CPS behaviors through test-
ing and monitoring.
In [ 22 ], authors introduce a novel approach employing
multiagent federated reinforcement learning to devise a se-
cure incentive mechanism in intelligent CPS. While fed-
erated learning addresses data privacy concerns in CPS,
ensuring ef ficient incentive mechanisms remains crucial.
Deep reinforcement learning is explored as a solution for
long-term incentivization amidst dynamic environments.
However , the heterogeneity of CPS devices poses a chal-
lenge, af fecting the conver gence rate of existing single-
agent reinforcement learning. The proposed multiagent
learning-based mechanism addresses this issue by approx-
imating stationarity in federated learning with heteroge-
neous CPS. The approach formulates the secure commu-
nication and data resource allocation problem as a Stackel-
ber g game and models it as a partially observable Markov
decision process to handle device heterogeneity . A mul-
tiagent federated reinforcement learning algorithm is de-
vised to ef ficiently learn allocation policies, mitigating pol-
icy evaluation variances caused by device interactions with-
out compromising privacy .
In [ 7 ], the authors propose a deep federated learning
framework to enhance secure points of interest (POI) mi-
croservices in CPS. This framework aims to improve data
security by isolating the cloud center from accessing user
data on edge nodes. Through interactive training between
the cloud center and edge nodes, reliable deep-learning-
based models are pre-trained on edge nodes, and parameter
updating is coordinated via federated learning. The pro-
posed approach is evaluated using real-world POI-related
datasets, demonstrating optimal scheduling performance
and practical utility .
2.1 Discussion
As shown in T able 1 , numerous studies have made signif-
icant contributions towards improving security consider -
ations (e.g., intrusion detection) and enhancing resilience
(e.g., self-adaptation) in CPS environments. However , de-
spite their contributions, limitations arise when considering
real-world deployment, particularly regarding security , pri-
vacy , and resource constraints.
Limitations of Existing Appr oaches:
– Centralized Machine Learning: While ef fective in
controlled settings, centralized ML architectures con-
centrate sensitive data in a central repository , mak-
ing them susceptible to security breaches such as sin-
gle point of failure vulnerabilities or insider threats.
Additionally , the continuous transmission of data to a
central server for training and updates incurs substan-
tial communication overhead, hindering scalability in
lar ge-scale CPS deployments.
– Federated Machine Learning: Although FML of fers
advantages in data privacy and communication ef fi-
ciency by keeping training data on local devices, its
decentralized nature presents challenges in resource-
constrained environments. Limited processing power ,
memory , and battery life on sensor devices can hinder
ef fective local model training, hindering widespread
adoption in CPS settings where ef ficient resource uti-
lization is paramount.
Advantages of our appr oach:
Our SML-based approach serves as a promising solution
that addresses the limitations of both centralized ML and
FML. By partitioning the training process between client
devices and a central server , our approach of fers several
advantages:
– Enhanced Data Privacy: Sensitive data remains
lar gely on local devices, minimizing the risk of expo-
sure through breaches in centralized storage.
– Reduced Communication Overhead: Only a portion
of the training data needs transmission to the server ,
significantly reducing communication costs compared
to centralized approaches.
– Ef ficient Resource Utilization: Local devices perform
computations on smaller datasets, alleviating the bur -
den on resource-constrained CPS nodes. This makes
SML well-suited for lar ge-scale deployments in CPS
environments.
Before introducing the novel contribution of this paper ,
we briefly present our previous work on the multilayer ar -
chitecture for adaptive CPS in the next section. Under -
standing these prior developments is crucial for contextu-
alizing our current approach, which builds upon this foun-
dation.

Ef ficient V anilla Split Learning for Privacy-Preserving … Informatica 48 (2024) 167–180 171
3 Our pr evious works
3.1 Pr oposal of an ar chitectur e for CPS
In our previous architecture for self-adaptive CPS [ 3 ], we
designed a comprehensive framework comprising essen-
tial software modules to ensure CPS functionalities. These
modules encompassed resource and data management, pro-
cess planning, and the transformation of machines into self-
aware, self-learning, and self-reconfiguring entities. Illus-
trated in Fig. 1 , our architecture consisted of distinct layers:
Physical layer : Serving as the foundation, this layer en-
compassed the local assembly of machines connected to the
Internet via the OPC-UA protocol, facilitating standardized
communication between dif ferent units. It primarily fea-
tured sensors responsible for collecting signals from ma-
chines and actuators for translating electrical signals into
physical movement.
Data/Resour ce Pr ocessing Layer : This layer facilitated
the collection of data from diverse sources, directly inter -
acting with data producers at the physical layer and data
storage in the Edge/Cloud. Modules within this layer in-
cluded the conditioner , data management, resource man-
agement, planning process, monitoring and control, and re-
quest management.
Data Storage Layer : Serving as the storage backbone
for the CPS, this layer was distributed between the Edge
and the Cloud, depending on the nature of the data and the
processing time required.
Learning Application Layer : At the forefront of adap-
tive functionalities, this layer facilitated resource alloca-
tion, QoS analysis, process optimization, predictive main-
tenance, and fault detection. Here, various ML techniques
such as regression, classification, clustering, and reinforce-
ment learning were employed. These models were trained
meticulously on a comprehensive dataset, combining his-
torical knowledge with real-time information to make pre-
dictions and informed decisions, driving the system’ s adap-
tive capabilities.
3.2 Integration of FML into our
ar chitectur e
Our centralized learning based architecture exhibited com-
mendable accuracy and precision. However , it encountered
constraints related to data security , privacy issues, and com-
munication overhead. These limitations highlighted the
need for an alternative strategy that could retain the advan-
tages of precise predictions while addressing the drawbacks
associated with centralized data handling and computation.
Our work in [ 4 ] addressed these limitations by incor -
porating FML into the architecture. This FML-based ap-
proach distributes processing and intelligence to the net-
work’ s edge, enabling local data analysis and decision-
making. By using FML, the architecture facilitates collab-
oration among edge devices to train ML models without re-
quiring centralized data collection. This decentralized ap-
proach of fers several advantages: it preserves data privacy ,
reduces communication overhead, and enhances system re-
silience, all while maintaining the high accuracy and preci-
sion achieved by the initial centralized architecture.
3.3 Practical implementation and
performance evaluation
T o validate the ef ficacy and real-world applicability of both
versions of our architecture, we implemented them in a
practical context as described in [ 4 , 5 ]. This application
aligns with the realm of failure prediction u sing supervised
and unsupervised learning algorithms. In this real-world
scenario, our system leveraged the power of supervised and
unsupervised learning algorithms to predict failures in a dy-
namic environment such as CPS.
In [ 5 ], we detailed the dif ferent ML algorithms used for
implementing the principle of fault prediction in CPS. Ex-
periments demonstrated the ef fectiveness of our approach
in fault prediction, with achieved accuracy surpassing 95%.
In [ 4 ], we detailed the training process in our FML-
based architecture. In this experiment, we utilized the same
dataset as in our previous work, enabling a direct compari-
son of the obtained results. This dataset will also be utilized
in this paper for comparison purposes.
In this paper , our goal is to significantly improve upon
our previous work in the context of resource-constrained
CPS by adopting the SML paradigm. The shift to SML ar -
chitecture is motivated by the need to address challenges
related to data privacy , security , and resource constraints,
which are particularly pertinent in CPS environments. Our
objective is to maintain the accuracy and precision of the
learning model while enabling more ef ficient utilization of
resources and greater resilience in dynamic operating con-
ditions.
4 The pr oposed SML-based
ar chitectur e
In our previous FML-based architecture, the collabora-
tive training process is distributed between the server and
clients. Although this approach ensures data privacy ,
it demands relatively higher computational resources on
the client side, which might be a constraint in resource-
constrained environments, such as CPS. The decentraliza-
tion of training in FML, while preserving privacy , can pose
challenges for devices with limited computational capabil-
ities.
Recognizing the importance of deploying machine learn-
ing models in CPS scenarios, where resource constraints are
prevalent, we transition to SML to address these challenges
ef fectively . In SML, the training process is shifted predom-
inantly to the server side, aligning with the characteristics
of resource-constrained devices. This shift allows for more
ef ficient utilization of available resources, making SML a
suitable alternative for applications in CPS.

172 Informatica 48 (2024) 167–180 N. Azeri et al.
Figure 1: The proposed Multi-layer architecture for CPS

Ef ficient V anilla Split Learning for Privacy-Preserving … Informatica 48 (2024) 167–180 173
The following subsections delve into the Split Learning
configuration used in our proposed SML-based approach,
as well as its key components and operational dynamics,
highlighting its advantages in addressing the challenges
posed by resource-constrained environments.
4.1 Split learning configurations
SML encompasses various configurations tailored to spe-
cific needs and constraints. Three common configurations
include [ 19 ]:
– Simple V anilla Configuration for Split Learning: In
this configuration, each client trains a partial deep net-
work up to a designated layer , known as the cut layer .
The outputs from this layer are forwarded to a central
server for further model training without exposing raw
data. The server aggregates gradients and sends them
back to clients for local backpropagation. This setup
minimizes data exposure while facilitating collabora-
tive model training.
– U-shaped Configurations for Split Learning without
Label Sharing: These configurations mitigate the need
for label sharing among clients while preserving data
privacy . The network architecture is wrapped around
at the end layers of the server ’ s network, and outputs
are sent back to clients.
– V ertically Partitioned Data for Split Learning: This
configuration enables collaborative model training
across multiple institutions with dif ferent data modal-
ities without sharing raw data. Each institution trains
a partial model up to its designated cut layer , and out-
puts are concatenated and forwarded to a central server
for further training.
For resource-constrained CPS, simplicity , ef ficiency ,
and privacy are critical considerations. The V anilla con-
figuration of split learning aligns with these requirements
for the following reasons:
– Simplicity and Ef ficiency: V anilla Split Learning pri-
oritizes a simple implementation, minimizing both
computational and communication overhead. In
resource-constrained CPS environments, where com-
putational resources and network bandwidth are lim-
ited, simplicity is essential for ef ficient model training
and inference.
– Data Privacy Preservation: By keeping raw data lo-
cal to each client and only sharing model updates with
the central server , V anilla split learning ensures robust
data privacy . This decentralized approach minimizes
the risk of data breaches or privacy violations, which
is crucial in CPS applications handling sensitive infor -
mation.
– Scalability and Adaptability: V anilla split learning’ s
decentralized nature makes it highly scalable and
adaptable to diverse CPS deployments. Clients can
operate independently , enabling seamless integration
with edge devices. This scalability facilitates the ex-
pansion of CPS deployments without compromising
performance or security .
– Flexibility in Model Customization: V anilla split
learning allows clients to customize their local model
architectures and training data to suit specific CPS
requirements. This flexibility enables adaptation to
varying environmental conditions and application do-
mains, enhancing the overall resilience and ef fective-
ness of the CPS.
Figure 2: The proposed V anilla SML-based architecture
4.2 Ar chitectur e components
In this subsection, we outline the key components of our
V anilla SML-based architecture and illustrate how they op-
erate together . W e focus on the enhancements introduced
compared to our previous architecture presented in Figure
1 . As illustrated in Figure 2 , the architecture comprises
three main entities: local data collectors, edge aggregators,
and the main server .
Local Data Collectors:
Local Data Collectors play a crucial role in managing
data within a group or federation. They gather informa-
tion from various sources, including industrial equipment,
smartphones, IoT devices, sensors, and other endpoints.
These Local Data Collectors prioritize data security and
privacy . Often, they achieve this by aggregating or sum-
marizing the collected data before forwarding it to the next

174 Informatica 48 (2024) 167–180 N. Azeri et al.
processing stage, the Edge Aggregator . In essence, Local
Data Collectors act as intermediaries, preparing and trans-
mitting relevant information while ensuring adherence to
privacy and security constraints.
Edge Aggr egators:
In our architecture, the Edge Aggr egator operates as the
client-side component responsible for managing the client-
side model (M
c
). In this collaborative learning setup, each
client, represented by an Edge Aggr egator , possesses a
unique local dataset (D
i
). Notably , the Edge Aggr egator
also plays a role in collecting and managing data from lo-
cal collectors, which include a diverse range of devices such
as industrial equipment, smartphones, IoT devices, sensors,
and other endpoints.
The client-side model (M
c
) is initialized with random
weights, and local training is conducted on the initial lay-
ers of the neural network, up to a designated cut layer that
separates the client-side and server -side segments.
During training iterations, forward propagation gener -
ates ”smashed data” up to the cut layer , representing ac-
tivations. This smashed data is securely transmitted to the
central server for further processing through the remaining
layers of the global model. The Edge Aggr egator manages
the client-side model (M
C
), orchestrates the training pro-
cess, and actively collects data from local collectors to con-
tribute to the collaborative learning paradigm.
Main server:
The Main server functions as the central server in our
V anilla SML architecture, orchestrating the collaborative
training process across multiple Edge Aggregators. In our
case, the server communicates with clients in a sequen-
tial manner to ensure a structured learning process. The
sequential communication involves interacting with each
client in a sequence (Client 1, Client 2, ..., Client n) for both
forward and backward passes (see figure 2 ). This approach
aims to enhance the learning process by iteratively refining
the global model through multiple interactions.
In our architecture, the communication between the Main
server and Edge Aggregators is a key aspect of the learn-
ing process. During the forward pass, smashed data (acti-
vations from the split layer , also known as the cut layer)
is transmitted from the client-side network to the server ,
allowing the global model to process the data through its
remaining layers. The backward pass involves transmit-
ting gradients generated at the server ’ s first layer back to
the Edge Aggregators, contributing to the refinement of the
client-side model.
4.3 Collaborative training in our
ar chitectur e
In our V anilla SML-based architecture, the primary objec-
tive is collaborative model training, wherein the server col-
laborates with each client in sequence for model training.
Each client possesses an individual local dataset denoted as
D
i
.
In our architecture, the training process comprises a
Client (C) and a server (S). A global model, denoted as
M
global
, is created, which consists of two distinct parts:
M
c
(Client-side Model) and M
s
(Server -side Model). Al-
gorithm 1 outlines the behavior of each client within our ar -
chitecture. TheM
c
is responsible for processing the initial
layers of the neural network, conducting local training on
the client’ s unique dataset, and generating ”smashed data”
after forward propagation up to a designated cut layer . On
the other hand, the M
s
, residing on a central server , uti-
lizes the smashed data received from the client to complete
forward propagation on the remaining layers of the global
model. The server side process is described in Algorithm
2 . The server conducts forward propagation on the server -
side model (M
s
) using the received smashed data to obtain
to minimize a loss function : L
i
( smash_data) . This is done
by the formula:
M
s
= ar gmin
M
L
i
( smash_data) (1)
After the training phase, the gradients▽ L
i
of the loss
function L
i
with respect to the model’ s parameters are
calculated. These gradients are determined using the
smash _data and the current model parameters, as shown
in formula 2 .
▽ L
i
=▽ Loss(M
s
, smash_data) (2)
This process involves computing the loss function, en-
gaging in backpropagation, and updating its own weights
until reaching the cut layer . The gradients corresponding to
the smashed data are then communicated back to the client.
This collaborative training process between theM
c
and the
M
s
iterates until conver gence, contributing to the refine-
ment of theM
global
.
5 Experimental evaluation
5.1 Data collection
The dataset, sourced from Kaggle and described in [ 1 1 ],
represents a synthetic dataset reflecting real predictive
maintenance scenarios. Figure 3 provides an overview of
the dataset columns, showcasing its structure. The dataset
consists of data points with the following main features:
– Pr oduct ID: Each product is assigned a unique identi-
fier consisting of a letter denoting the product quality
variant (L for low , M for medium, and H for high) and
a variant-specific serial number .
– Air T emperatur e [K]: Generated through a random
walk process and subsequently normalized to have a
standard deviation of 2 K around the mean temperature
of 300 K.

Ef ficient V anilla Split Learning for Privacy-Preserving … Informatica 48 (2024) 167–180 175
Algorithm 1 Client-Side Algorithm
Requir e: M
c
: Client-side Model with random weights
D
c
: Local data on the client side
num _epochs : Number of training epochs
Ensur e: Smashed data for the server
1: for epoch← 1 tonum _epochs do
2: M
c
← TrainLocalModel(M
c
,D
c
)
3: smash _data← M
c
.get _smash _data()
4: SendToServer(smash _data)
5: gradients _from _server← ReceiveFromServer()
6: M
c
.backward (gradients _from _server)
7: end for
Algorithm 2 Server -Side Algorithm
Requir e: M
s
: Initial server -side model
clients : List of clients
Ensur e: Updated gradients
1: for client in clients do
2: for epoch← 1 tonum _epochs do
3: smash _data← receive _from _client(client)
4: predictions← M
s
.forward (smash _data)
5: loss← calcul _loss(predictions,smash _data)
6: M
s
.backward (loss)
7: send _to _client(client,M
s
.get _gradients())
8: end for
9: end for
Figure 3: Dataset structure
– Pr ocess T emperatur e [K]: Produced using a random
walk process, normalized to a standard deviation of 1
K, and added to the air temperature plus 10 K.
– Rotational Speed [rpm]: Derived from a power of
2860 W and superimposed with normally distributed
noise.
– T or que [Nm]: T orque values follow a normal distri-
bution around 40 Nm with a standard deviation of 10
Nm, ensuring no negative values.
– T ool W ear [min]: T ool wear duration varies based on
product quality variants, with high, medium, and low
variants adding 5, 3, and 2 minutes, respectively , to the
total tool wear during the process.
Additionally , in this dataset, we have the ’T ar get’ label
that indicates whether the machine has failed in this partic-
ular datapoint. If the process fails, the ’T ar get’ label is set
to 1.
The choice of this dataset enables a meaningful compari-
son of results with our previous work. Throughout the data
preparation phase, privacy considerations were paramount,
aligning with the privacy-preserving principles of SML. T o
assess our proposal, we divided the dataset among three
separate clients, ensuring a representative distribution of
data and facilitating a comprehensive evaluation of our
SML-based approach. In line with this, we utilized a syn-
thetic dataset representing real predictive maintenance sce-
narios, divided among three clients. Each client’ s model

176 Informatica 48 (2024) 167–180 N. Azeri et al.
was trained locally using T ensorFlow Keras, with a sequen-
tial server -client communication model.
The subsequent sections elaborate on the implementa-
tion of our V anilla SML approach using T ensorFlow Keras,
providing insights into the architecture and training pro-
cess. W e then analyze the conver gence of loss and accu-
racy during SML training, examining how these metrics
evolve over time and evaluating the stability and ef ficacy
of our approach. Following this, we present a compara-
tive analysis, wherein we juxtapose the performance of our
SML-based approach with existing methods, highlighting
its advantages and contributions in the realm of resource-
constrained CPS deployments.
5.2 V anilla SML implementation with
T ensorFlow Keras
Central to the SML paradigm are the client models, which
encapsulate domain-specific knowledge while training on
local datasets. Utilizing the T ensorFlow Keras API [ 6 ],
we instantiate client models with architectures tailored to
accommodate the dimensions and characteristics of local
data. Each client model comprises multiple densely con-
nected layers, incorporating activation functions such as
ReLU to introduce non-linearity and facilitate model con-
ver gence. In our case, client models are created using the
create_client_model() function, which defines the ar -
chitecture and compiles the models.
Client models are trained iteratively across multi-
ple epochs using the fit() function from the T en-
sorFlow Keras API, with each epoch encompassing
forward and backward propagation facilitated by the
train_on_batch() method. This method updates model
parameters using stochastic gradient descent. T raining pa-
rameters such as batch size and learning rate are meticu-
lously tuned using functions such as compile() and fit()
to optimize conver gence while mitigating computational
overhead. In our case, the compile() function configures
the model for training, specifying the Adam optimizer for
ef ficient gradient descent and the binary cross-entropy loss
function. This loss function quantifies the dif ference be-
tween model predictions and ground truth labels during bi-
nary classification tasks.
During training, client models exclusively access and
learn from their respective local datasets, preserving data
privacy and confidentiality . This is facilitated by functions
such as fit() , which train the model on local data without
sharing it externally . Following the client model construc-
tion, a central server model is also created, adhering to the
principles of V anilla SML. W e utilize the Sequential()
constructor from the T ensorFlow Keras API to define the
server model’ s architecture. This architecture is designed
to be compatible with the client models, using the func-
tion Dense() to add densely connected layers. The server
model acts as the central nexus for knowledge aggregation
within the SML paradigm. During the training process,
client models train locally and extract relevant information
(e.g., gradients) after a designated cut layer . These are then
sent to the server for aggregation. In our case, the server
uses the functions get_weights() and set_weights()
to retrieve these gradients and incorporate them into its
own model, facilitating the collaborative learning process.
This iterative exchange of gradients between clients and the
server continues until a conver gence criterion is met.
Following construction, the server model is compiled
using the compile() function to prepare for training and
knowledge aggregation, entailing specifying optimization
algorithms, loss functions, and optional evaluation metrics.
In our implementation, the Adam optimizer is employed for
ef ficient gradient descent optimization, while binary cross-
entropy serves as the loss function for binary classifica-
tion tasks. Evaluation metrics, including accuracy , pro-
vide insights into model performance and conver gence dur -
ing training and aggregation phases, facilitated by functions
such as evaluate() .
5.3 Convergence analysis of loss and
accuracy during SML training
As elucidated earlier , the training process hinges on a syner -
gistic collaboration between the server and the clients. This
collaborative endeavor unfolds across multiple epochs,
aiming to minimize the loss function and maximize the
model’ s accuracy .
The central element of the training process is the compu-
tation of the loss value on the server side. This loss value
serves as a crucial metric, quantifying the disparity between
the model’ s predictions and the actual tar get values in the
training dataset. A smaller loss value indicates a closer
alignment between the model’ s predictions and the smashed
data points, signifying an improved predictive capability .
This approach ensures that the training process not only
refines the model’ s parameters but also steers it towards a
state where its predictions better capture the underlying pat-
terns within the training data.
Figure 4 depicts the relationship between the number of
epochs and both the loss and accuracy for each client. It is
evident that the number of training epochs plays a crucial
role in shaping the loss values. During the initial training
epochs, a substantial reduction in the loss is observed, indi-
cating significant improvements as the model learns from
client data. As training progresses (epochs = 70), the loss
stabilizes at a minimum value, representing the optimal per -
formance achievable with the given data and model archi-
tecture. T rying to add more epochs shows that the loss lev-
els stabilize, indicating that the model has reached a point
where further training doesn’ t result in substantial improve-
ment, signifying conver gence. Conver gence signifies that
the model has ef fectively captured patterns and relation-
ships present in the training data.
Regarding accuracy , the graph illustrates dynamic
changes during the training process. As the number of train-
ing epochs increases, accuracy steadily rises until it attains
a stable value, reaching this stability at around 70 epochs.

Ef ficient V anilla Split Learning for Privacy-Preserving … Informatica 48 (2024) 167–180 177
Figure 4: Loss and accuracy conver gence for each client
This stability suggests that the model has learned compre-
hensively , performing optimally on the provided dataset.
The attainment of a consistent accuracy level indicates that
the model has successfully learned and adapted to the un-
derlying patterns in the training data. Notably , at each
epoch, the server leverages the smashed data received from
the clients to enhance the global model, contributing to the
continuous improvement in accuracy . After an initial in-
crease during the early epochs, the accuracy values for all
three clients stabilized at an average of 0.9756.
5.4 Comparative analysis
T able 2 presents the results obtained by the approach pro-
posed in this paper alongside those obtained in our previous
work [ 4 , 5 ]. It’ s important to note that we utilized the same
dataset for the sake of comparison. Like presented in T able
2 , it is striking that both the centralized and SML techniques
show similar degrees of accuracy , with both achieving ac-
curacies up to 97%. This parity in accuracy underscores the
viability of SML as a competitive alternative to centralized
learning approaches.
T o further assess the performance of decentralized ap-
proaches in resource-constrained environments, we delve
into a comparative analysis between FML and SML ap-
proaches. While T able 2 provides insights into the accuracy
metrics across dif ferent ML approaches, additional met-
rics are necessary to evaluate their ef ficiency in resource-
T able 2: Comparison of accuracy metrics across dif ferent
ML approaches
ML approach Accuracy
SML 0.9756
FML 0.9544
Centralized 0.9785
limited contexts. Specifically , we focus on the ”Learning
time for each client” metric, which sheds light on the com-
putational resource utilization of each approach.
Maintaining consistent experimental configurations for
FML and SML ensures a fair comparison. This includes us-
ing identical client configurations and dataset distributions
across clients. Such uniformity enables an accurate evalua-
tion and comparison of the performance of FML and SML
under resource-constrained environments.
As presented in Figure 5 , the results obtained from
the experiments reveal notable dif ferences in the learning
time for each client between FML and SML. In resource-
constrained environments, where ef ficient resource utiliza-
tion is paramount, SML demonstrates a clear advantage.
The learning time for each client in SML is significantly
shorter compared to FML, indicating that clients utilizing
SML consume fewer computational resources.
Specifically , the experimental results are summarized in
T able 3 , which shows the comparison of learning time be-
tween FML and SML for each client.
On average, across all clients, the SML approach
achieves a reduction in learning time of approximately
41%. These results highlight the ef ficiency of SML in re-
ducing learning time, which is critical for real-time applica-
tions in CPS. By significantly lowering the computational
load on clients, SML not only enhances learning speed but
also conserves resources, making it a highly suitable ap-
proach for resource-constrained environments.
Figure 5: T otal learning time for each client in FML and
SML
Consequently , SML emer ges as a more ef ficient and suit-

178 Informatica 48 (2024) 167–180 N. Azeri et al.
Client Learning T ime in FML Learning T ime in SML Reduction (%)
Client 1 12.2 seconds 9.5 seconds 22.13%
Client 2 1 1.3 seconds 6.0 seconds 46.90%
Client 3 1 1.3 seconds 5.0 seconds 55.75%
T able 3: Comparison of learning time between FML and SML for each client
able approach for resource-constrained CPS deployments.
Its ability to optimize computational resource utilization
makes it a compelling alternative to FML in such environ-
ments. By ef fectively addressing the challenges associated
with limited computational resources, SML showcases its
potential to enhance the performance and adaptability of
CPS systems operating under resource constraints.
However , it is important to acknowledge some poten-
tial limitations of the SML approach. Firstly , like in FML,
SML relies on a central server for model aggregation, which
could become a single point of failure. Implementing fault-
tolerant mechanisms or exploring decentralized alternatives
could mitigate this risk. Secondly , despite reducing com-
munication overhead compared to centralized approaches,
SML still requires transmitting model updates. Optimiz-
ing this further or managing it in networks with limited
bandwidth remains an area for improvement. Further -
more, dif ferences in local data distributions and training
processes could lead to model inconsistencies, highlighting
the importance of ensuring conver gence and model stability
across dif ferent clients. Moreover , the overall performance
of SML can still be af fected by network latency , especially
in environments with poor connectivity . Strategies to min-
imize the impact of network delays should be explored.
6 Conclusion
This paper has introduced a novel SML-based architecture
specifically designed to address the dual challenges of d ata
privacy and resource constraints in CPS. Our proposed ar -
chitecture utilizes the distributed learning power of SML
to enable real-time system adaptation based on local sensor
data, while simultaneously preserving data privacy .
Our experimental evaluation highlights the ef fectiveness
of the proposed SML-based architecture. W ith an achieved
accuracy of 97%, SML demonstrates competitive perfor -
mance when compared to centralized learning approaches,
surpassing the accuracy achieved by FML. Notably , the
learning time for each client in SML is shorter than FML,
making it a practical choice for resource-constrained CPS
deployments.
Our future work will focus on further refining the SML
architecture. W e will explore dif ferent configurations, such
as U-shaped or vertically partitioned approaches, to poten-
tially improve ef ficiency and accuracy . Additionally , we
aim to investigate fault-tolerant mechanisms and decen-
tralized alternatives to reduce the risk of a single point of
failure. Furthermore, we will optimize the communication
overhead and manage model updates more ef ficiently , es-
pecially in networks with limited bandwidth.
Acknowledgements
The author would like to thank the anonymous reviewers
for their valuable comments and suggestions, which were
helpful in improving the paper .
Refer ences
[1] Rania Salih Ahmed, Elmustafa Sayed Ali Ahmed,
and Rashid A Saeed. Machine learning in cyber -
physical systems in industry 4.0. In Artificial intel-
ligence paradigms for smart cyber -physical systems ,
pages 20–41. IGI global, 2021.
[2] Odey Alshboul, Rabia Emhamed Al Mamlook, Ali
Shehadeh, and T ahir Munir . Empirical exploration
of predictive maintenance in concrete manufacturing:
Harnessing machine learning for enhanced equipment
reliability in construction project management. Com-
puters & Industrial Engineering , page 1 10046, 2024.
doi:10.1016/j.cie.2024.110046 .
[3] Nabila Azeri, Ouassila Hioual, and Ouided Hioual.
T owards an approach for modeling and architecting
of self-adaptive cyber -physical systems. In 2022 4th
International Confer ence on Pattern Analysis and In-
telligent Systems (P AIS) , pages 1–7. IEEE, 2022. doi:
10.1109/pais56586.2022.9946921 .
[4] Nabila Azeri, Ouided Hioual, and Ouassila Hioual.
Enhancing self-adaptive cyber -physical systems us-
ing federated machine learning. In T ACC 2023 : 3r d
T unisian-Algerian Joint Confer ence on Applied Com-
puting , pages 108–1 19. ceur -ws.or g, 2023.
[5] Nabila Azeri, Zeinb Zouikri, Meriem Rezgui, Ouided
Hioual, and Ouassila Hioual. Fault prediction us-
ing supervised and unsupervised learning algorithms
in cyber physical systems. In 2022 2nd Interna-
tional Confer ence on New T echnologies of Informa-
tion and Communication (NTIC) , pages 1–6. IEEE,
2022. doi:10.1109/ntic55069.2022.10100404 .
[6] Multi framework deep learning API. Keras: The
high-level api for tensorflow , 2023. https://www.
tensorflow.org/guide/keras .
[7] Zhiwei Guo, Keping Y u, Zhihan Lv , Kim-
Kwang Raymond Choo, Peng Shi, and Joel JPC

Ef ficient V anilla Split Learning for Privacy-Preserving … Informatica 48 (2024) 167–180 179
Rodrigues. Deep federated learning enhanced se-
cure poi microservices for cyber -physical systems.
IEEE W ir eless Communications , 29(2):22–29, 2022.
doi:10.1109/mwc.002.2100272 .
[8] Xianting Huang, Jing Liu, Y ingxu Lai, Beifeng Mao,
and Hongshuo L yu. Eefed: Personalized federated
learning of execution&evaluation dual network for
cps intrusion detection. IEEE T ransactions on In-
formation For ensics and Security , 18:41–56, 2022.
doi:10.1109/tifs.2022.3214723 .
[9] Sangjun Kim and Kyung-Joon Park. A survey on
machine-learning based security design for cyber -
physical systems. Applied Sciences , 1 1(12):5458,
2021. doi:10.3390/app11125458 .
[10] Beibei Li, Y uhao W u, Jiarui Song, Rongxing Lu, T ao
Li, and Liang Zhao. Deepfed: Federated deep learn-
ing for intrusion detection in industrial cyber–physical
systems. IEEE T ransactions on Industrial Informat-
ics , 1 7(8):5615–5624, 2020. doi:10.1109/tii.
2020.3023430 .
[1 1] Stephan Matzka. Explainable artificial intelligence
for predictive maintenance applications. In 2020 thir d
international confer ence on artificial intelligence for
industries (ai4i) , pages 69–74. IEEE, 2020. doi:
10.1109/ai4i49448.2020.00023 .
[12] Hossein Mohammadi Rouzbahani, Hadis Karimipour ,
Abolfazl Rahimnejad, Ali Dehghantanha, and Gau-
tam Srivastava. Anomaly detection in cyber -physical
systems using machine learning. Handbook of big
data privacy , pages 219–235, 2020. doi:10.1007/
978- 3- 030- 38557- 6_10 .
[13] Felix O Olowononi, Danda B Rawat, and Chunmei
Liu. Resilient machine learning for networked cyber
physical systems: A survey for machine learning se-
curity to securing machine learning for cps. IEEE
Communications Surveys & T utorials , 23(1):524–
552, 2020. doi:10.1109/comst.2020.3036778 .
[14] Mutaz R yalat, Hisham ElMoaqet, and Marwa Al-
Faouri. Design of a smart factory based on cyber -
physical systems and internet of things towards indus-
try 4.0. Applied Sciences , 13(4):2156, 2023. doi:
10.3390/app13042156 .
[15] Arman Sar golzaei, Carl D Crane, Alireza Abbaspour ,
and Shirin Noei. A machine learning approach for
fault detection in vehicular cyber -physical systems. In
2016 15th IEEE International Confer ence on Machine
Learning and Applications (ICMLA) , pages 636–640.
IEEE, 2016. doi:10.1109/icmla.2016.0112 .
[16] Zakir Ahmad Sheikh, Y ashwant Singh, Pradeep Ku-
mar Singh, and Kayhan Zrar Ghafoor . Intelligent
and secure framework for critical infrastructure (cps):
Current trends, challenges, and future scope. Com-
puter Communications , 193:302–331, 2022. doi:
10.1016/j.comcom.2022.07.007 .
[17] Rama Mercy Sam Sigamani. Adoption of ma-
chine learning with adaptive approach for secur -
ing cps. In Handbook of Resear ch on Machine
and Deep Learning Applications for Cyber Security ,
pages 388–415. IGI Global, 2020. doi:10.4018/
978- 1- 6684- 6291- 1.ch061 .
[18] Theocharis Theocharides, Muhammad Shafique,
Jungwook Choi, and Onur Mutlu. Guest editorial:
Robust resource-constrained systems for machine
learning. IEEE Design & T est , 37(2):5–7, 2020.
doi:10.1109/mdat.2020.2971201 .
[19] Praneeth V epakomma, Otkrist Gupta, T ristan
Swedish, and Ramesh Raskar . Split learning for
health: Distributed deep learning without sharing raw
patient data. arXiv pr eprint arXiv:1812.00564 , 2018.
[20] Chathurika S W ickramasinghe, Kasun Amarasinghe,
Daniel L Marino, Craig Rieger , and Milos Manic. Ex-
plainable unsupervised machine learning for cyber -
physical systems. IEEE Access , 9:131824–131843,
2021. doi:10.1109/access.2021.3112397 .
[21] Kok-Seng W ong, Manh Nguyen-Duc, Khiem Le-
Huy , Long Ho-T uan, Cuong Do-Danh, and Danh Le-
Phuoc. An empirical study of federated learning
on iot-edge devices: Resource allocation and hetero-
geneity . arXiv pr eprint arXiv:2305.19831 , 2023.
[22] Minrui Xu, Jialiang Peng, Brij B Gupta, Jiawen Kang,
Zehui Xiong, Zhenni Li, and Ahmed A Abd El-
Latif. Multiagent federated reinforcement learning
for secure incentive mechanism in intelligent cyber–
physical systems. IEEE Internet of Things Journal ,
9(22):22095–22108, 2021. doi:10.1109/jiot.
2021.3081626 .
[23] Chong Zhang, Xiao Liu, Xi Zheng, Rui Li, and
Huai Liu. Fenghuolun: A federated learning based
edge computing platform for cyber -physical systems.
In 2020 IEEE international confer ence on pervasive
computing and communications workshops (PerCom
W orkshops) , pages 1–4. IEEE, 2020. doi:10.1109/
percomworkshops48775.2020.9156259 .

180 Informatica 48 (2024) 167–180 N. Azeri et al.