Skip to main content

Google Cloud Storage

Overview​

This document outlines the process for submitting data stored in Google Cloud Storage (GCS) to AlphaSOC for analysis. AlphaSOC analyzes log files by identifying security threats and detecting anomalies.

To transfer data, set up the following GCS resources:

  • GCS bucket that stores the data and triggers event notifications sent to a Pub/Sub topic.
  • Pub/Sub topic that receives notifications when files are uploaded to the GCS bucket.
  • Push subscription that forwards Pub/Sub topic messages to AlphaSOC.

Prerequisites​

Before you begin, ensure you have the following:

  • Google Cloud Storage (GCS) bucket with log delivery configured,
  • gcloud CLI installed,
  • roles/pubsub.editor IAM role to create a Pub/Sub topic and a push subscription.

Creating a Pub/Sub Topic​

In the Google Cloud console, navigate to Pub/Sub > Topics and click Create topic. Enter a Topic ID and click Create.

Creating a Push Subscription​

Go to the AlphaSOC console, click Credentials on the sidebar, and copy your AlphaSOC Ingestion Token. You will need this token later to configure the Pub/Sub push subscription.

alphasoc-ingestion-token

Once you copy the token, go to the Google Cloud console. Navigate to the Pub/Sub > Subscriptions page and click Create subscription.

Enter a Subscription ID and select your previously created topic. Select Push as the delivery type and enter the following URL: https://api.alphasoc.net/gcs/importFromCloudStorage?access_token=<ALPHASOC_TOKEN>, replacing <ALPHASOC_TOKEN> with your AlphaSOC Ingestion Token.

push-subscription

Next, configure the retry policy. Set the minimum backoff to 60s and the maximum backoff to 600s.

retry-policy

Granting AlphaSOC Access to the GCS Bucket​

In the Google Cloud console, go to the Storage > Cloud Storage > Buckets page and select your log bucket. Select the Permissions tab and click Grant access.

In the New principals field, enter data-import@alphasoc-io.iam.gserviceaccount.com. Select the Storage Object Viewer role from the dropdown.

gcs-grant-access

Click Save to grant access.

Creating Notifications​

Use Google Cloud CLI to create notifications for the bucket. Enter the following command, replacing:

gcloud storage buckets notifications create gs://<YOUR_LOGS_BUCKET> --topic=<TOPIC_NAME> --event-types=OBJECT_FINALIZE

Example:

gcloud storage buckets notifications create gs://your-bucket --topic=projects/test/topic/name --event-types=OBJECT_FINALIZE

Your GCS bucket is now configured to send notifications to the Pub/Sub topic when new files are uploaded. The push subscription will forward these notifications to AlphaSOC for processing.

Further Reading​