S3 Lifecycle Rules

Automate object transitions, expirations, and cost optimization with lifecycle policies

S3 lifecycle rules automate object management, helping you optimize storage costs and comply with data retention policies.

Lifecycle Rules Overview

What Lifecycle Rules Do

Transition: Move objects between storage classes
Expiration: Delete objects after a specified time
Cleanup: Remove incomplete multipart uploads and expired delete markers

Basic Lifecycle Configuration

Apply lifecycle configuration

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration file://lifecycle.json

Get current lifecycle configuration

aws s3api get-bucket-lifecycle-configuration --bucket my-bucket

Delete lifecycle configuration

aws s3api delete-bucket-lifecycle --bucket my-bucket

Rule Structure

Complete lifecycle rule example

{
  "Rules": [
    {
      "ID": "MyLifecycleRule",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "logs/"
      },
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        }
      ],
      "NoncurrentVersionTransitions": [
        {
          "NoncurrentDays": 30,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      },
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 90
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      }
    }
  ]
}

Filter Types

Apply to objects with a specific prefix:

Prefix filter

{
  "ID": "LogsRule",
  "Status": "Enabled",
  "Filter": {
    "Prefix": "logs/"
  },
  "Expiration": {
    "Days": 90
  }
}

Apply to objects with specific tags:

Tag filter

{
  "ID": "TempFilesRule",
  "Status": "Enabled",
  "Filter": {
    "Tag": {
      "Key": "type",
      "Value": "temporary"
    }
  },
  "Expiration": {
    "Days": 7
  }
}

Combine prefix and tags:

Combined filter

{
  "ID": "CombinedRule",
  "Status": "Enabled",
  "Filter": {
    "And": {
      "Prefix": "data/",
      "Tags": [
        {"Key": "env", "Value": "dev"},
        {"Key": "archive", "Value": "true"}
      ]
    }
  },
  "Transitions": [
    {"Days": 30, "StorageClass": "GLACIER"}
  ]
}

Apply to all objects (empty filter):

All objects filter

{
  "ID": "AllObjectsRule",
  "Status": "Enabled",
  "Filter": {},
  "AbortIncompleteMultipartUpload": {
    "DaysAfterInitiation": 7
  }
}

Empty filter applies to ALL objects in the bucket. Use carefully.

Storage Class Transitions

Transition Rules

Transition Constraints

Minimum 30 days before transitioning to IA classes
Minimum 90 days before transitioning to Glacier Instant
Transitions happen at midnight UTC
Small objects (under 128KB) may not transition to IA classes

Transition Waterfall

Progressive transition example

{
  "Rules": [
    {
      "ID": "ProgressiveArchive",
      "Status": "Enabled",
      "Filter": {"Prefix": "data/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "STANDARD_IA"},
        {"Days": 90, "StorageClass": "GLACIER_IR"},
        {"Days": 180, "StorageClass": "GLACIER"},
        {"Days": 365, "StorageClass": "DEEP_ARCHIVE"}
      ]
    }
  ]
}

Supported Transitions

S3 Standard
    ↓
S3 Intelligent-Tiering
    ↓
S3 Standard-IA / One Zone-IA
    ↓
S3 Glacier Instant Retrieval
    ↓
S3 Glacier Flexible Retrieval
    ↓
S3 Glacier Deep Archive

Objects cannot transition backwards (e.g., Glacier to Standard-IA).

Object Expiration

Delete objects after N days:

Expire after 90 days

{
  "ID": "ExpireOldLogs",
  "Status": "Enabled",
  "Filter": {"Prefix": "logs/"},
  "Expiration": {
    "Days": 90
  }
}

Delete objects on a specific date:

Expire on specific date

{
  "ID": "ProjectCleanup",
  "Status": "Enabled",
  "Filter": {"Prefix": "project-alpha/"},
  "Expiration": {
    "Date": "2024-12-31T00:00:00Z"
  }
}

For versioned buckets, expiration creates delete markers:

Expiration with versioning

{
  "ID": "VersionedExpiration",
  "Status": "Enabled",
  "Filter": {},
  "Expiration": {
    "Days": 365
  },
  "NoncurrentVersionExpiration": {
    "NoncurrentDays": 90
  },
  "ExpiredObjectDeleteMarker": true
}

Noncurrent Version Management

For versioned buckets, manage old versions:

Transition old versions

{
  "ID": "ArchiveOldVersions",
  "Status": "Enabled",
  "Filter": {},
  "NoncurrentVersionTransitions": [
    {
      "NoncurrentDays": 30,
      "StorageClass": "STANDARD_IA"
    },
    {
      "NoncurrentDays": 90,
      "StorageClass": "GLACIER"
    }
  ]
}

Delete old versions

{
  "ID": "DeleteOldVersions",
  "Status": "Enabled",
  "Filter": {},
  "NoncurrentVersionExpiration": {
    "NoncurrentDays": 90
  }
}

Keep only the most recent N versions:

Keep last 3 versions

{
  "ID": "KeepRecentVersions",
  "Status": "Enabled",
  "Filter": {},
  "NoncurrentVersionExpiration": {
    "NoncurrentDays": 1,
    "NewerNoncurrentVersions": 3
  }
}

This deletes noncurrent versions older than 1 day, keeping only the 3 most recent noncurrent versions.

Delete Markers and Multipart Uploads

Clean Up Delete Markers

Remove delete markers when they're the only version remaining:

Remove expired delete markers

{
  "ID": "CleanupDeleteMarkers",
  "Status": "Enabled",
  "Filter": {},
  "Expiration": {
    "ExpiredObjectDeleteMarker": true
  }
}

Abort Incomplete Multipart Uploads

Clean up failed uploads to save storage:

Abort incomplete uploads

{
  "ID": "AbortIncompleteUploads",
  "Status": "Enabled",
  "Filter": {},
  "AbortIncompleteMultipartUpload": {
    "DaysAfterInitiation": 7
  }
}

Hidden Costs

Incomplete multipart uploads are often overlooked. They consume storage and incur costs until cleaned up.

Common Lifecycle Patterns

Log Retention

Log lifecycle policy

{
  "Rules": [
    {
      "ID": "LogRetention",
      "Status": "Enabled",
      "Filter": {"Prefix": "logs/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "STANDARD_IA"},
        {"Days": 60, "StorageClass": "GLACIER"}
      ],
      "Expiration": {"Days": 365}
    }
  ]
}

Development Environment

Dev environment cleanup

{
  "Rules": [
    {
      "ID": "DevCleanup",
      "Status": "Enabled",
      "Filter": {
        "Tag": {"Key": "env", "Value": "dev"}
      },
      "Expiration": {"Days": 30}
    }
  ]
}

Data Lake Tiering

Data lake lifecycle

{
  "Rules": [
    {
      "ID": "RawDataTiering",
      "Status": "Enabled",
      "Filter": {"Prefix": "raw/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "INTELLIGENT_TIERING"}
      ]
    },
    {
      "ID": "ProcessedDataArchive",
      "Status": "Enabled",
      "Filter": {"Prefix": "processed/"},
      "Transitions": [
        {"Days": 90, "StorageClass": "STANDARD_IA"},
        {"Days": 365, "StorageClass": "GLACIER"}
      ]
    }
  ]
}

Compliance Archive

7-year retention for compliance

{
  "Rules": [
    {
      "ID": "ComplianceRetention",
      "Status": "Enabled",
      "Filter": {"Prefix": "compliance/"},
      "Transitions": [
        {"Days": 90, "StorageClass": "GLACIER_IR"},
        {"Days": 180, "StorageClass": "DEEP_ARCHIVE"}
      ],
      "Expiration": {"Days": 2555}
    }
  ]
}

Backup Management

Backup lifecycle

{
  "Rules": [
    {
      "ID": "DailyBackups",
      "Status": "Enabled",
      "Filter": {"Prefix": "backups/daily/"},
      "Expiration": {"Days": 30}
    },
    {
      "ID": "WeeklyBackups",
      "Status": "Enabled",
      "Filter": {"Prefix": "backups/weekly/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "GLACIER"}
      ],
      "Expiration": {"Days": 90}
    },
    {
      "ID": "MonthlyBackups",
      "Status": "Enabled",
      "Filter": {"Prefix": "backups/monthly/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "GLACIER"},
        {"Days": 180, "StorageClass": "DEEP_ARCHIVE"}
      ],
      "Expiration": {"Days": 365}
    }
  ]
}

Intelligent-Tiering Configuration

Configure Intelligent-Tiering archive access tiers:

Configure archive tiers

aws s3api put-bucket-intelligent-tiering-configuration \
  --bucket my-bucket \
  --id archive-config \
  --intelligent-tiering-configuration '{
    "Id": "archive-config",
    "Status": "Enabled",
    "Tierings": [
      {
        "Days": 90,
        "AccessTier": "ARCHIVE_ACCESS"
      },
      {
        "Days": 180,
        "AccessTier": "DEEP_ARCHIVE_ACCESS"
      }
    ]
  }'

Testing Lifecycle Rules

Enable the Rule

Set Status to Enabled in your configuration.

Wait for Processing

Lifecycle rules run once per day at midnight UTC. Wait at least 24 hours.

Check Object Storage Class

Verify storage class

aws s3api head-object \
  --bucket my-bucket \
  --key my-object.txt \
  --query StorageClass

Use CloudWatch Metrics

Monitor NumberOfObjects and BucketSizeBytes by storage class.

Lifecycle Rule Limits

Limit	Value
Maximum rules per bucket	1,000
Maximum tags per rule	10
Minimum transition days (to IA)	30
Minimum size for IA transition	128KB

Troubleshooting

Common Issues

Rules not applying: Wait 24+ hours; rules run daily at midnight UTC
Small objects not transitioning: Objects under 128KB may not transition to IA classes
Transition constraints: Check minimum days between storage classes
Filter conflicts: Multiple rules with overlapping filters can cause unexpected behavior

Debug with S3 Inventory

Set up inventory to track storage classes:

Create inventory configuration

aws s3api put-bucket-inventory-configuration \
  --bucket my-bucket \
  --id storage-class-inventory \
  --inventory-configuration '{
    "Id": "storage-class-inventory",
    "IsEnabled": true,
    "Destination": {
      "S3BucketDestination": {
        "Bucket": "arn:aws:s3:::inventory-bucket",
        "Format": "CSV",
        "Prefix": "inventory/"
      }
    },
    "Schedule": {"Frequency": "Daily"},
    "IncludedObjectVersions": "Current",
    "OptionalFields": ["StorageClass", "LastModifiedDate"]
  }'

Best Practices

Lifecycle Rule Guidelines

Start with longer retention - It's easier to shorten than recover deleted data
Use Intelligent-Tiering when access patterns are unknown
Always clean up multipart uploads - Add this rule to every bucket
Test in development first - Validate rules before applying to production
Document your policies - Use meaningful rule IDs
Monitor with CloudWatch - Track storage by class
Review regularly - Adjust rules as needs change
Consider compliance - Some data may have mandatory retention periods

Cost Estimation

Estimate savings from lifecycle policies:

Get current storage by class

aws cloudwatch get-metric-statistics \
  --namespace AWS/S3 \
  --metric-name BucketSizeBytes \
  --dimensions Name=BucketName,Value=my-bucket Name=StorageType,Value=StandardStorage \
  --start-time $(date -v-7d +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date +%Y-%m-%dT%H:%M:%SZ) \
  --period 86400 \
  --statistics Average

Approximate savings by storage class (vs Standard):

Standard-IA: ~40% cheaper
One Zone-IA: ~50% cheaper
Glacier Instant: ~68% cheaper
Glacier Flexible: ~78% cheaper
Glacier Deep Archive: ~95% cheaper

Next Steps

Storage Classes

Understand storage tiers

Versioning

Version management

Security

Access control

S3 CLI Reference

Complete CLI commands

S3 Lifecycle Rules

Storage Classes

Versioning

Security

S3 CLI Reference

On this page