The allure of serverless architecture has always been the promise of focus: focus on code, focus on business logic, and focus on the user, without the cognitive overhead of managing infrastructure. Google Cloud Run epitomized this “serverless dream” by allowing developers to run stateless containers that scale to zero and back up in milliseconds.
However, for a long time, that dream had a caveat – statelessness. While stateless applications are the gold standard for microservices, real-world enterprise workloads often require a “persistent soul.” Whether it’s legacy Content Management System (CMS) applications, data-intensive processing, or shared configuration files, the need for robust storage has been the final hurdle for migrating complex apps to Cloud Run.
The Evolution of Cloud Run Storage
As the platform has matured, the storage landscape for Cloud Run has expanded significantly. We are no longer limited to simple, transient container file systems. Today, Cloud Run supports a diverse array of volume types, allowing architects to choose the right tool for their specific IO requirements and persistence needs.
Choosing Your Volume Strategy
When architecting your Cloud Run service, consider the following decision matrix:
- Need for Security? Use Secrets. This ensures sensitive data never touches the disk or environment variables, mounting directly into the container’s file system.
- Need for Speed? Use In-Memory volumes. If your data can be regenerated or is purely for short-term caching, the RAM-backed speed is unbeatable.
- Need for Scale and Object Storage? Use Cloud Storage bucket mounts. It is perfect for serving images or long-term archival of logs where true POSIX compliance isn’t a strict requirement.
- Need for Shared Persistence? Use Filestore. This is the choice for any application that expects a “real” hard drive shared across multiple scaling instances of your container.
- Need for Large Temporary Files? Use the Ephemeral disk (Preview). When your processing job exceeds the standard memory-backed limit, the ephemeral disk provides the breathing room necessary for large-scale data manipulation.
Why Filestore for Cloud Run is the Game Changer
While each volume type has its place, Cloud Filestore represents the most significant bridge for enterprise workloads moving to serverless. Unlike other storage options, Filestore provides a fully managed, POSIX-compliant file system.
This means your application doesn’t need to learn a new API (like the Cloud Storage API) to read and write files. It uses standard file operations (open, read, write, close). For applications that require multiple instances to read and write to the same shared directory simultaneously, Filestore is the “soul” that makes the serverless dream a reality.
Detailed Real-World Use Case
Integrating Filestore with Cloud Run isn’t just a technical curiosity; it solves complex architectural challenges across various industries.
The Parallel Batch Data Processing (“Scatter-Gather”) pattern is widely considered one of the canonical, textbook architectural patterns for pairing serverless compute with shared network file storage.
While serverless platforms like Cloud Run excel at elastic, on-demand compute, parallel batch workloads often choke on storage bottlenecks when relying on object storage abstractions or local temporary buffers. Google Cloud Filestore (NFS) serves as the critical enabler for this architecture by providing true, POSIX-compliant Network Attached Storage that scales concurrently across hundreds of container instances. Unlike Cloud Storage FUSE, which incurs latency overhead from REST API calls and lacks atomic file locking, Filestore delivers sub-millisecond metadata operations and native support for simultaneous ReadWriteMany access. This allows parallel Cloud Run array tasks to stream input chunks, perform deterministic task partitioning, and write intermediate results directly to a shared filesystem without file corruption, REST API rate-limiting, or memory bloat. By decoupling active processing state from compute, Filestore unlocks true HPC-grade parallel processing in a completely serverless environment.
The diagram above presents high-performance, parallelized image watermarking batch processing architecture on Google Cloud. It leverages Cloud Run Jobs with Hybrid Storage Mounting (Google Cloud Storage via GCS FUSE + Google Cloud Filestore via NFS) and Direct VPC Egress to optimize both throughput and large-scale batch processing.
The pipeline includes:
- Ingestion Phase: Raw image assets are uploaded to the primary source-of-truth GCS Input Bucket.
- Initialization & Multi-Volume Mounting: The Cloud Run Job initializes with multi-volume mounts:
* GCS FUSE Mounting: Mounts input and output buckets.
* Filestore NFS Mounting: Mounts Filestore instance. - Parallel Batch Processing:
* Fetch: Container reads raw image from input GCS bucket.
* Process: Copies image to Filestore for high-IOPS manipulation (watermark rendering, format conversion). - Output & Archival:
* The watermarked image is written directly to the GCS Output Bucket.
* Task-specific temporary scratch directory on Filestore NFS is automatically purged.
* Structured JSON metrics are streamed to Cloud Logging.
Try It Yourself: Clone the Demo Repo!
To help you and your teams see this pipeline in action, I’ve packaged all the configuration files and scripts into a repository. Feel free to clone and try it!
Conclusion
The “stateless” label for serverless is becoming a thing of the past. With the integration of Filestore and the expansion of Cloud Run volume types, we can finally build applications that scale with serverless agility while maintaining the persistent heart required by modern enterprise software.
Whether you are migrating a legacy application or building a new data-heavy microservice, the storage tools are now in your hands. It is time to let your Cloud Run services dream bigger.
The Serverless Dream with a Persistent Soul: Supercharging Cloud Run with Filestore was originally published in Google Cloud – Community on Medium, where people are continuing the conversation by highlighting and responding to this story.
Source Credit: https://medium.com/google-cloud/the-serverless-dream-with-a-persistent-soul-supercharging-cloud-run-with-filestore-a898410b1bbf?source=rss—-e52cf94d98af—4
About the Author
