🤖 AI Summary
This work proposes a secure and transparent egress solution for sandboxed workloads to ensure multi-tenant isolation and fair resource allocation. By constructing a defense-in-depth architecture that integrates eBPF-based packet filtering, GENEVE overlay networking, and distributed egress proxies, the system enables policy-driven network access control with low overhead. A novel two-tier policy enforcement mechanism is introduced: at the lower layer, eBPF enforces precise bandwidth rate-limiting using the Earliest Departure Time (EDT) algorithm, while the upper layer provides protections against connection exhaustion and port depletion. Deployed across all Snowflake regions, the system supports petabyte-scale data transfers and low-latency external integrations, meeting the stringent security and performance requirements of large-scale production environments.
📝 Abstract
Snowpark enables data engineering and AI/ML workloads in Snowflake by executing user-defined functions in secure sandboxes. Many of these workloads require external connectivity to access cloud APIs, external databases, or feature stores, creating a dependability challenge: how to provide transparent network access while preserving strict multi-tenant isolation and resource fairness. This paper presents Secure Network Access in Snowpark (SNAS), a production architecture for secure external communication from sandboxed workloads. SNAS combines Extended Berkeley Packet Filter (eBPF) packet filtering, Generic Network Virtualization Encapsulation (GENEVE) overlay networks, and distributed egress proxies for policy-driven egress control with low overhead. We describe the design, deployment, and measured production behavior of SNAS, including an eBPF-based bandwidth limiter using the Earliest Departure Time (EDT) algorithm, dual-tier policy enforcement, and safeguards for connection limiting and port exhaustion. SNAS is deployed across all Snowflake regions and supports large-scale production workloads including petabyte-scale data transfer and latency-sensitive external integrations.