Amazon S3 Monitoring

Amazon S3 provides a reliable cloud storage platform to store and retrieve objects at any scale.

With Site24x7's S3 integration you can collect S3 logs, service level storage and request metrics on an individual bucket level. You can also, run periodic checks on critical S3 objects from global locations to monitor functionality and response time.

Table of contents

• Setup and configuration
• Policies and permissions
• Polling frequency
• Security findings
• Supported performance metrics
  • Storage metrics
  • Request metrics
  • Replication metrics
• Configuration details
• Threshold Configuration
• Forecast
• Checking S3 Objects
• Amazon S3 Object monitoring
  • Prerequisites
  • Add S3 Object Monitor
  • Discovering S3 Objects
  • Selecting S3 Objects
  • Configuring S3 Object Monitor
• Amazon S3 Folder monitoring
  • Prerequisites
  • Why S3 Folder monitoring?
  • Discovering and adding S3 Folder Monitors

Setup and configuration

• If you haven't already, enable site24x7 access to your AWS account via key based or IAM role base authentication
• In the Integrate AWS Account page, select the S3 checkbox under the Services to be discovered field.

Policies and permissions

Site24x7 requires the following permissions to discover and monitor your S3 buckets. Learn more.

• "s3:GetObjectAcl",
• "s3:GetEncryptionConfiguration",
• "s3:GetLifecycleConfiguration",
• "s3:GetBucketTagging",
• "s3:ListAllMyBuckets",
• "s3:GetBucketVersioning",
• "s3:GetBucketAcl",
• "s3:GetBucketLocation",
• "s3:GetBucketPolicy",
• "s3:GetReplicationConfiguration",
• "s3:GetBucketLogging"
• "s3:GetObjectAcl",
• "s3:ListBucket",
• "s3:GetBucketLocation"

Polling frequency

Storage metrics are collected on a day to day basis. Request and data transfer metrics are collected as per the poll frequency set (1 minute to a day). Learn more.

Security findings

Fortify your Amazon S3 instances against cyber attacks by integrating Amazon GuardDuty and S3 in a single console. GuardDuty findings for Amazon S3 can be grouped according to its severity level. The group by feature lets you sort the severity categories and lists the corresponding Rules Package Name and Comments if any.

In addition to adding these threshold configurations for your monitored S3 instances, you can also choose to set thresholds and get notified for:

• Security findings based on the severity level like high, low, medium, informational, or total count under the GuardDuty threshold configuration.
• Accelerator configurations like GPU memory utilization, accelerator memory usage, and accelerator utilization.

Supported performance metrics

Storage metrics

Request metrics

Enable request metrics for your S3 buckets, using the Amazon S3 management console to monitor operations on your objects and buckets.

Replication metrics

Configuration details

The following inventory data is collected for your monitored S3 bucket.

Threshold Configuration

To configure thresholds for your Site24x7 - S3 Bucket integrated monitor:

1. Nnavigate to Admin > Configuration Profiles > Threshold and Availability.
2. Click Add Threshold Profile.
3. Select S3 Bucket from the Monitor Type drop-down menu, and provide an appropriate name in the Display Name field.
4. The supported metrics are displayed in the Threshold Configuration section. You can set threshold values for all the metrics mentioned above and configure to receive notification when the S3 Bucket is public by toggling the Notify when Bucket is Public option to Yes.
5. Click Save.

Forecast

Estimate future values of the following performance metrics and make informed decisions about adding capacity or scaling your AWS infrastructure.

• Bucket Size
• Number of Objects

Checking S3 Objects

You can check for the presence of S3 objects at the bucket level and set thresholds for the same. Be notified if the defined object is missing from the S3 bucket.

Amazon S3 Object monitoring

Use Site24x7's prebuilt API monitoring functionality to run periodic checks on S3 object URLs to ensure its functionality. At the configured interval, Site24x7 sends out HTTP requests to the object URL from each of the configured location and validates the result.

Prerequisites

• The Amazon S3 monitoring integration must be enabled.
• The state of the S3 bucket monitor should be active.

Add S3 Object Monitor

This section describes how to discover, add and configure checks for your object API endpoints:

• In the Site24x7 web console, select AWS > Monitored AWS account.
• From the drop down menu, select Add S3 Endpoint.

Discovering S3 Objects

This sections describes how you can discover specific object URL endpoints from your S3 bucket:

• Choose an S3 bucket from the drop down (multiple selection is not possible.)
• Type the proper Regex to match the object URLs in the selected S3 bucket. Append .* as a suffix or as a prefix to your input string to only opt-in specific object URLs from the selected bucket. For example, you can discover only the objects in a particular path or directory using a Regex pattern like (//img.site24x7static.com/images/.*) or you can discover objects of a certain type using a Regex pattern like (.png.*).
• Next click on, Discover Objects

Selecting S3 Objects

All the object URLs matching your Regex pattern would get listed below. Here, you can select the endpoints that you wish to monitor by clicking on the checkbox and choosing Add Selected Objects

Configuring S3 Object Monitor

To add the S3 object endpoint monitor, you need to specify values for the checks configuration.

• Display Name: Provide an appropriate name for the check
• S3 Object URL: The selected S3 object endpoint URLs would be listed here. You can remove an endpoint form the list by clicking, deselecting and saving the form.
• Check frequency: Select any option from 1, 5, 10, 20, 25, 30(minutes) or 1, 2 (hours). For example if you choose 1 minute, each selected geographic location would try to reach your endpoint every 1 minute.
• Connection timeout: Specify a connection time between 1-45 seconds. Checks that do not generate a response within the period will be termed as failures.
• Prefer IPv6: If you are monitoring dual stack object endpoints, move the toggle to Yes.
• Monitoring locations: Create a new location profile or select or edit an exiting profile. The endpoint would receive requests from the applicable locations. Learn more .
• Monitor Groups: Logically group your monitors for better management. Create a new monitor group or add the end point monitor to an exiting monitor group by clicking on the
• Dependent on monitor: Alerts to the S3 object monitor will be suppressed based on the DOWN status of your dependent resource.

Specify the following details for HTTP configuration:

• HTTP Method: Specify the HTTP method to be used, you can choose between GET or HEAD
• User Agent: Type in a user agent if you want to emulate a specific browser.

Specify the following details for response validation:

• Should contain string(s): Fill in a string whose presence in the response indicates failure
• Should not contain string(s): Fill in a string whose non presence in the response indicates failure
• Case sensitive: Switch the toggle to Yes if the response is case sensitive
• Should match regular expression: Type in a Regex pattern whose presence in response indicates failure.

Specify the following details to setup configuration profiles:

• Threshold and Availability: Configure values for each supported attribute and get alerted when the metric data point crosses the value you have defined. You can either create a new threshold profile or select/edit the default threshold profile
• Notification Profile: Configure when and who needs to be notified in case of threshold violation or failure. You can either create a new notification profile or select/edit an existing profile. Refer here to create a customized notification profile.
• User Alert Group: Select the Alert Group. To add multiple users in a group, refer here
• Tags: User-defined key-value pairs associated with the object would get discovered and automatically assigned.
• IT Automation: You can automatically execute operational tasks on AWS resource in response to alert events. You can create an automation profile and associate to the object endpoint monitor and execute it whenever the monitor goes to down/trouble/up/any status change or whenever an attribute value changes relative to a threshold.
• Third Party Integration: You can integrate the monitor with a third-party alerting service to facilitate improved incident management. If you haven't setup any integrations yet, navigate to ”Admin > Third Party Integration” to create one. Learn more.
• Click Save.