Cribl
This guide provides step-by-step instructions on how to integrate Cribl Stream with AlphaSOC using a custom S3 destination via MinIO. By following this short guide, you'll be able to send your logs from any source to AlphaSOC for analysis.
Prerequisites
- A running instance of Cribl Stream (Cloud or on-premise)
- One or more sources setup and receiving telemetry data
- Access to the AlphaSOC Analytics Engine
- Credentials to the AlphaSOC destination (S3)
Setting up AlphaSOC as a Cribl destination
Log in to your Cribl Stream instance with a username with admin privileges or a custom role that allows you to use Sources, Destinations and QuickConnect.
Navigate to Manage > Routing > QuickConnect
Click Add Destination and select MinIO.
Provide the necessary details (omitted parameters imply default values):
General Settings
| Parameter | Value |
|---|---|
| Output ID | AlphaSOC |
| MinIO endpoint | https://s3.alphasoc.net |
| MinIO bucket name | events |
| Key Prefix | cribl |
| Data Format | Raw |
| Partitioning expression | `${C.Time.strftime(_time ? _time : Date.now()/1000, '%Y/%m/%d')}/\${__inputId}` |
| Compress (default) | gzip |
| File name prefix expression | `CriblOut` |
| File name suffix expression | `.${C.env["CRIBL_WORKER_ID"]}.${__format}${__compression === "gzip" ? ".gz" : ""}` |
Authentication
Obtain your Access Key and Secret Key from the AlphaSOC Workspace. Log in, navigate to Credentials at the bottom of the left menu, and in the S3 Server Credentials section, copy the keys or click New credentials to generate a new set.
Note: Ensure to enclose each key within backticks (``). Example format: `[Your-Access-Key-Here]`.
The configuration windows should look as follows:
Save the configuration and commit/deploy the changes.
Connecting to an Existing Source
Navigate to Routing -> QuickConnect in the Cribl Stream web interface.
Drag a line between the Source you want to connect and the newly created AlphaSOC destination. You can do this for multiple sources. Choose Passthru to leave the data unaltered or select a Pipeline or Pack if you wish to perform any operation on the data.
Connection Configuration Options
- PassThru: This will send the logs directly without any modifications.
- Pipeline/Pack: If you have specific processing or enrichment needs based on the source, select the appropriate pipeline(s).
Save the configuration and commit/deploy the changes.
Optional - Setting up a Pack or Pipelines
If you've chosen to use a pack for data enrichment or transformation then you can assign a pack as follows:
Testing the Integration
First, check the status tab of the configuration panel of AlphaSOC destination and verify that the destination is working properly as below:
Then under the Test tab, select a sample file and click Run Test
You should get a Success message as shown above.
You can now start sending some test logs to Cribl Stream from your chosen source or use the internal datagen source to generate some data. Sample logs can also be provided if needed.
Verify that the logs are successfully received in the AlphSOC destination by checking the Live Data tab on the configuration panel.
Check the AlphaSOC Analytics Engine console to ensure that the logs are being processed.
Debugging
Before checking the logs it is important to increase the verbosity of the channel in the group settings.
Make note of (or remember), the channel name (available in Logs tab for the destination configuration panel)
Go to Settings > Logging > Levels and search for the channel name. Typically, if you named the destination as AlphaSOC, searching as illustrated below should bring up the one channel:
Change the logging level to either debug (or even silly for even more verbosity) then return to the Logs tab of the configuration panel to inspect the logs for further information on the problem.