Personal Embedding Transformation for Privacy-Preserving Split Learning on Textual Data

초록

Split learning is a collaborative learning paradigm designed to reduce client-side computational load while providing inherent privacy advantages by splitting up a model among the client and the server. However, it faces a critical vulnerability: an honest-but-curious server can reconstruct the original client data from the intermediate representation of the client model, called smashed data, which is transmitted to the server. Prior defenses typically inject random noise into smashed data to reduce privacy leakage. Although noise-based defenses enhance privacy, the presence of noise hinders server model training, resulting in reduced overall model performance. To effectively defend privacy leakage against original data reconstruction attacks targeting smashed data while minimizing model performance degradation, we propose the Personal Embedding Transformation and Contrastive Representation Separation module. The Personal Embedding Transformation module randomly applies a set of differentiable transformation functions to the embedding dimensions within the client model. These transformations diversify and reshape the embedding space in a client-specific manner. Contrastive Representation Separation reorganizes the transformed embeddings through contrastive learning, clustering representations of the same class while pushing apart those of different classes. The combination of these mechanisms produces client-specific embedding spaces that distort sensitive information while retaining class-level discriminative features for downstream tasks. The proposed mechanisms enhance privacy guarantees in the split learning setting while mitigating the severe downstream model utility degradation typically associated with noise-based defenses. Experiments on large-scale text classification demonstrate that our framework achieves strong protection against reconstruction attacks while incurring negligible degradation in model utility and computational efficiency. Specifically, our method introduces only a ∼2.56% increase in client-side inference latency, confirming its practical feasibility. In particular, the proposed method reduces reconstruction similarity by 34% in cosine score and lowers text-level overlap metrics such as BLEU and ROUGE by more than 33% compared to noise-based defense, indicating that adversarially reconstructed texts exhibit significantly reduced semantic similarity to the original inputs. © 2013 IEEE.

키워드