Let's Deploy a Data Application

It's honestly easier than you think

The premise for below is that you have a locally functional data application using Streamlit that you would like to deploy in the cloud for others to have access to. There is a lot of mystery around how to do that for Data Teams. I think its time to skill up to understand this step in the Full Data Stack. All code will be available in a Github along with some screen shots of billing info after I deployed on Cloud Run.

Let’s Get Started:

Today we are going to deploy a fully functional Streamlit application onto GCP using Cloud Run and Cloud Build. This simple application is meant to mimic a data analysis tool you could use to understand streaming behavior data from a company like Hulu. The data is all synthetic and static in the app itself, for now. This application will sit in GCP Cloud Run and will be connected to a Github repo. We will setup CICD for the application so that it will listen for any changes to the Github repo main branch and deploy a new build each time there is.

Here’s what it will look like:

Here’s what you need:

1. A Streamlit application in a Github repo: Click here for code and readme

2. Astral UV to import libraries: Get Astral here

3. Docker installed on your local computer: Get Docker here

4. The Google Cloud SDK installed on your local computer: Get gcloud here

5. A GCP account (just use you gmail email account) and make sure you have permissions to create Cloud Run projects (may need to enable Cloud Run and Cloud Build API).

6. Create a project in the GCP account

A Quick Note About Security:

This quick and dirty way to deploy an app is missing a couple things at the top that you would want to consider before having this app become external facing:

• We aren’t doing any authorization in this tutorial. In GCP you can use internal permissions to control who has access to this app. For a fully external app, you’d wan to add some auth layer in case you would like to manage who has access to the application.

What is Cloud Run?

From the official documentation HERE

Cloud Run is a managed compute platform that lets you run containers directly on top of Google's scalable infrastructure.

You can deploy code written in any programming language on Cloud Run if you can build a container image from it. In fact, building container images is optional. If you're using Go, Node.js, Python, Java, .NET Core, Ruby, or a supported framework you can use the source-based deployment option that builds the container for you, using the best practices for the language you're using.

So essentially, what Cloud Run does is it takes a Docker image of your application and makes a function that will call it to spin it up when users try to access the URL. The nice thing about Cloud Run is that it autoscales down to 0 when no one is accessing the URL. So if no one goes to your app, then you aren’t going to get charged.

This differs from another deployment strategy, like Kubernetes, in that there is no pod to manage. You essentially get the DevOps handled by GCP and just work on improving the application code and pushing PR’s.

Some other options are things like Vercel, however for Data Teams they are probably going to have other services like Cloud Storage and Data Warehouses to connect to so it’s nice to keep your deployment platform in the same place as those. You can also set up IAM authorization permissions in GCP which lets you control access. It’s just a nice to have and easy to deploy.

Let’s Go Through the Readme

OK, so with that high level out of the way let’s go through the readme I have in my linked Github account. For Streamlit apps, you have the option to run them locally in your browser using a virtual environment. If you are new to Streamlit, here’s how it works:

Local Development

1. Create the virtual environment:

   uv venv .venv
   source .venv/bin/activate  
   # On Windows: .venv\Scripts\activate

2. Install Dependencies:

   uv pip install -r requirements.txt

3. Run the Streamlit app locally:

   streamlit run Home.py

Once you’re comfortable running the app locally in a virtual environment, you now want to build a Docker image for the application then test the Docker image locally. Docker images represent a virtual machine that is not your computer, so testing the Docker image gives you a close proximation of how you app will actually function once it is deployed to the cloud:

Docker for Development

1. Build the Docker Image

   docker build --platform linux/amd64 -t streamlit-cohort-analysis .

2. Run in development mode:

   docker run -p 8501:8501 -e DEV_MODE=true streamlit-app

Now that you have tested a Docker image locally on your machine and things look correct, let’s build a Docker image specifically for Cloud Run. A little DevOps detail here is that when you deploy an app in a cloud vendor like Cloud Run, it is being stored on a server that has a certain OS and hardware set up. This set up might be different from your local computer, so you need to actually build a Docker image that is meant to be read by the cloud hardware. Cloud Run specifically uses a linux/amd64 OS and hardware setup. So we first make the Docker image for that:

1. For Mac with Apple Silicon or other ARM-based systems:

   docker build --platform linux/amd64 -t streamlit-cohort-analysis .

2. For x86/amd64 systems:

   docker build -t streamlit-cohort-analysis .

If you are running on a Mac, you won’t be able to run this particular Docker image locally because it is now set up to be run by your system. That is why we do the local Docker image first to get a lay of the land of how the app will perform. Once you create this Cloud Run specific app, we want to authorize gcloud and push to the GCP Artifact Registry. This is where Docker images are kept on GCP and be used by cloud services. We are using gcloud SDK so we can do this in our code editor terminal and introduce software best practices.

1. Authenticate with Google Cloud

   # Login to Google Cloud
   gcloud auth login

   # Configure Docker to use Google Cloud credentials 
   gcloud auth configure-docker

   # Set your project ID 
   gcloud config set project YOUR_PROJECT_ID

2. Tag and Push the Docker Image

   # Tag the image for Google Container Registry
   docker tag streamlit-cohort-analysis gcr.io/YOUR_PROJECT_ID/streamlit-cohort-analysis
   
   # Push the image to Google Container Registry
   docker push gcr.io/YOUR_PROJECT_ID/streamlit-cohort-analysis

3. Deploy to Cloud Run

   gcloud run deploy streamlit-cohort-analysis \
     --image gcr.io/YOUR_PROJECT_ID/streamlit-cohort-analysis \
     --platform managed \
     --allow-unauthenticated \
     --region us-central1 \
     --project YOUR_PROJECT_ID

The above deployment commands are going to create the Cloud Run app on the project you add to it, allow unauthenticated access (anyone can go to the public URL) and put it in the central1 region. You can change any of these as you see fit based on your own research and needs.

After the app is deployed and you don’t see any error messages you can jump over to Cloud Run and see the app available:

If you click on the application name you’ll be taken to a page that has metrics on the app usage. At the top is an “Edit Continuous Deployment” menu selection:

You’ll be taken to the “Triggers” section of Cloud Run. From here you’ll work through some steps to connect your Github repo to Cloud Run. If you are using your own Github and your own GCP account this should all be fairly straight forward. If you are trying to do all of this on your company’s GCP then you will probably be hit with some permissions denials and need to work with your IT team to get them. But I suggest trying to do all of this on your own end of things first. It gives you a great perspective when trying to pitch it as a solution to your broader org.

After you have the Github repo connected, if you merge a PR or update the main branch, then a new build for the app will be created and deployed on Cloud Run. It is an automated CICD just for you!

Let’s Talk About Costs

For the most part, deploying a small app like this is going to all fall into the Cloud Run free tier. Since Cloud Run autoscales to 0 when the app is not in use you aren’t charged for it running when you aren’t looking at it.

However, I did see costs incurred when I was updating the Github repo and the CICD was automatically kicking in. Over the first 24 of me getting the app up then doing a couple PR’s to fine tune the look I incurred $3.82. That’s obviously not much and I haven’t done any update to the Github main branch since and it’s been $0 cost.

So you can expect to get pinged a few dollars, of which you can see in your billing page on GCP. But if you are fine with that, then I highly suggest adding the CICD so you can get comfortable with how that works!

Conclusion:

The time has come for Data Teams to start “shifting right” in the data stack and up skilling on how an in house application can be deployed. In the new world of AI development it’s completely possible for a team to build their own custom BI tool and to deploy for internal use at a small cost to the company. All while removing DevOps overhead. Let’s start getting comfortable with these tools and take the bull by the horns.