PI-SAFE: Practical Privacy-preserving LLM Inference with Adversarial Fine-tuning for Optimized Utility

초록

Cloud-based Large Language Model (LLM) inference services typically require users to submit plain-text inputs, thereby posing severe privacy risks. Existing privacy-preserving paradigms are mostly task-specific and often necessitate pervasive modifications to the entire server-side model. This reliance introduces substantial computational overhead, elevated deployment costs, and an inevitable privacy-utility trade-off. In this paper, we propose PI-SAFE, a novel framework enabling privacypreserving LLM inference for both text generation and classification tasks, notably without requiring any alterations to the serverside model. By partitioning a pre-trained LLM between the client and the server, PI-SAFE facilitates collaborative inference via the transmission of obfuscated intermediate representations rather than raw text. To defend against reconstruction attacks, we introduce a Dual-constraint Adversarial Fine-Tuning (AdvFT) mechanism, which empowers the client to reshape the output feature distribution by covertly inserting adapter modules. To further counter powerful adversaries equipped with massive prior data who employ Deep Neural Networks (DNNs) for inverse fitting, we propose an enhanced framework, PI-SAFE+. This framework introduces a random prefix-based feature obfuscation mechanism. By employing a locally generated, fixed secret token sequence for feature fusion, PI-SAFE+ fundamentally disrupts the topological isomorphism of the feature mapping space. Extensive experiments demonstrate that, under equivalent privacy guarantees, PI-SAFE+ significantly outperforms existing baselines in inference utility. Crucially, it reduces the success rate of optimization-based reconstruction attacks to near zero and degrades attribute inference to random-guessing levels, exhibiting exceptional empirical defense and computational efficiency.

키워드