This extended abstract explores the integration of federated learning with deep transfer hashing for distributed prediction tasks, emphasizing resource-eficient client training from evolving data streams. Federated learning allows multiple clients to collaboratively train a shared model while maintaining data privacy - by incorporating deep transfer hashing, high-dimensional data can be converted into compact hash codes, reducing data transmission size and network loads. The proposed framework utilizes transfer learning, pre-training deep neural networks on a central server, and fine-tuning on clients to enhance model accuracy and adaptability. A selective hash code sharing mechanism using a privacy-preserving global memory bank further supports client fine-tuning. This approach addresses challenges in previous research by improving computational eficiency and scalability. Practical applications include Car2X event predictions, where a shared model is collectively trained to recognize trafic patterns, aiding in tasks such as trafic density assessment and accident detection. The research aims to develop a robust framework that combines federated learning, deep transfer hashing and transfer learning for eficient and secure downstream task execution.
The rapid growth of data and the increasing emphasis on privacy-preserving machine learning
techniques have spurred significant interest in federated learning (FL) [
FL is a distributed machine learning paradigm where multiple clients collaboratively train a shared model while keeping their data decentralized and secure. This approach is particularly beneficial for applications that require strict data protection and security measures. By combining federated learning with deep transfer hashing techniques, we aim to eficiently convert high-dimensional data into compact, low-dimensional hash codes, significantly decreasing data transmission size between the FL server and clients, reducing network transfer loads, and potentially improving client inference eficiency. Deep transfer hashing methods have proven to be highly efective in reducing the dimensionality of data while preserving its intrinsic structure. This capability is crucial for classification tasks, where the highdimensional nature of data often poses significant computational challenges. In this context, locality traffic scene under observation traffic input stream server feature extractor hashing layer server << feedback >> << update >> feature extractor hashing layer local memory bank
client << share >> sensitive hashing and learning to hash approaches have been widely used. However, traditional locality sensitive hashing methods require the construction and administration of numerous hash tables, which can be impractical for distributed optimisation tasks such as those observed in FL. Learning to hash potentially provides a more scalable and eficient solution by leveraging the powerful representation capabilities of deep learning models to learn complex hash functions in an end-to-end manner. To further enhance the performance of our proposed framework, we employ transfer learning by pre-training the DNN model on the high-performance server. This pre-trained model can then be fine-tuned on client devices using their local data streams, ensuring that the model adapts to the specific characteristics of each client’s data. This approach not only accelerates the training process but also improves the overall accuracy and generalization capability of the model.
We aim to integrate our approach in practical scenarios that involve raw or pre-processed data points
from monitoring sensors installed at various locations, such as trafic cameras and surveillance cameras
as seen in Figure 1. In these scenarios, models can learn to recognize patterns, objects, or anomalies.
For example, in a Car2X [
We consider a FL environment composed of a central, resource-heavy server tasked with network orchestration and multiple resource-restricted clients indexed by , where = 1, . . . as outlined in Fig. 1. Additionally, client inspects sample , seen only once, from the non-I.I.D. data stream at time , in which the occurrence of objects is not evenly distributed, using an arbitrary sensing device.
In the preparation phase, a task-specific deep neural network model is pre-trained on the server to
learn a task-specific hash function ℎ that maps an input to a binary code , facilitating the nearest
neighbor search used for prediction inference. A well-designed hash function should preserve the
relative distances between items in the original space, meaning that items close to a specific query
in Hamming space should also be close to the query in the original space [
Hash. Participating clients receive the pre-trained model and have on-demand access to the global
memory bank , utilizing these resources to perform localized fine-tuning with the sampled data set.
This process involves each client further optimizing the hash function ℎ to better adapt to their specific
data, thereby enhancing the model’s performance for their particular fine-tuning tasks. To achieve this,
we aim to employ a pointwise-based hashing method. Recent advancements typically construct the
classification loss within the Hamming space. Specifically, these methods generate a set of central hash
codes, each associated with a specific class label. The objective is to enforce the network outputs to
converge towards their respective hash centers using various loss terms, thereby ensuring that the hash
function preserves the relative distances between items efectively [
Adapt. An important consideration in deploying a FL model for critical real-world applications like Car2X is the phenomenon of concept drift, where the statistical properties of data points sampled from the client data stream change over time. This drift can result from evolving user preferences, seasonal variations, or other dynamic factors influencing the data distribution. To maintain the eficacy of our transferred hashing algorithm, it is crucial to evaluate and implement strategies for detecting concept drift. Integrating mechanisms to handle the changing data problem into our incremental hash code learning process ensures that the model adapts to new patterns and continues to generate accurate and relevant hash codes. Efective detection and adaptation techniques will help to maintain the performance and reliability of the model, even as the underlying data distributions change over time, raising Open Question 3.
Repeat. The adaptation on local clients is repeated until the global model converges. Updates received on the server (hash codes, model parameters) from the clients after each round of federated training must be integrated into both the server model and the global memory bank and thereby maintaining data integrity and data privacy, raising Open Question 4.
Overall, evaluation and benchmarking of the proposed method is essential, with a clear justification for its preference over existing techniques. A major challenge is the availability of ground truth data, especially to assess the model’s handling of concept drift, raising Open Question 5. Algorithm 1 Sample. Hash. Adapt. Repeat.
Require: Pre-trained hash function ℎ on client , unlabeled stream of samples , global memory bank 1: Initialize global memory bank 2: for each federated learning round do 3: Initialize = 1 4: Initialize = {} ◁ Initialize local hash code storage 5: for ∈ do ◁ Sample from data stream 6: = calculate_hash() ◁ Hash code generation 7: ← ∪ {} 8: ← + 1 9: end for 10: ℎ = adapt(, ) ◁ Adapt client model 11: Send update to server 12: end for ◁ Repeat until convergence 13: return Converged global learning model, educated global memory bank
In this work, we presented the integration of federated learning with deep transfer hashing methods for distributed classification and retrieval tasks. Our approach focuses on resource-aware client training from evolving data streams, leveraging transfer learning through pre-trained models on the FL server, and utilizing selective hash code sharing via a privacy-preserving global memory bank. This integration is supposed to eficiently convert high-dimensional data into compact hash codes, reducing network data transmission and improving client training and inference eficiency.
This concept paper outlines a foundational idea aimed at addressing challenges in data privacy,
computational eficiency, and scalability in FL prediction tasks. To underline the importance and
relevance of our research, we identified and outlined a real-world use-case that enhances several areas
of Car2X application. We seek to initiate further discussions and collaborations to refine these concepts
and advance privacy-preserving machine learning techniques. By attending this workshop, we hope to
gain educated insights from the discussions on the following open questions:
Open Question 1: What hash codes to include in the global memory bank (class prototypes) and
what is the best type and structure of the memory layout (map, tree, graph)?
Open Question 2: How can external memory banks improve distributed FL learning while sticking to
all FL constraints (data privacy, communication eficiency, . . . ) ?
Open Question 3: How to properly integrate the incremental aspect in deep transfer hashing to take
account for concept drift as proposed in [
MR is supported through the Bavarian HighTech Agenda, specifically by the Würzburg Center for Artificial Intelligence and Robotics (CAIRO) and the ProPere THWS scholarship.