The disparity in healthcare personnel expertise and medical resources across different regions of the world is a pressing social issue. Artificial intelligence technology offers new opportunities to alleviate this issue by empowering diagnostic and treatment capabilities in underdeveloped areas. Segment Anything Model (SAM), which excels in intelligent image segmentation, has demonstrated exceptional performance in medical monitoring and assisted diagnosis. Unfortunately, the huge computational and storage overhead of SAM poses significant challenges for deployment on resource-limited edge devices, especially in underdeveloped regions with limited equipment and computing power. Quantization is an effective solution for model compression; however, traditional methods rely heavily on original data for calibration, which raises widespread concerns about medical data privacy and security. In this paper, we propose a data-free quantization framework for SAM, called DFQ-SAM, which learns and calibrates quantization parameters without any original data, thus effectively preserving data privacy during model compression. Specifically, we propose pseudo-positive label evolution for segmentation, combined with patch similarity, to fully leverage the semantic and distribution priors in pre-trained models, which facilitates high-quality data synthesis as a substitute for real data. Furthermore, we introduce scale reparameterization to ensure the accuracy of low-bit quantization. We perform extensive segmentation experiments on datasets of various modalities such as CT and MRI, and DFQ-SAM consistently provides significant performance on low-bit quantization, e.g., 4-bit quantization results in only a 2.01% accuracy decrease in abdominal organ segmentation on AbdomenCT1K dataset. DFQ-SAM decouples the model deployment from real data, eliminating the need for data transfer in cloud-edge collaboration, thereby protecting sensitive data from potential attacks. By offloading complex medical analysis tasks to local nodes and employing privacy-preserving model compression, it enables secure, fast, and personalized healthcare services at the edge. This enhances system efficiency and optimizes resource allocation, and thus facilitating the pervasive application of artificial intelligence in worldwide healthcare, especially in remote or resource-limited regions.