AWSTemplateFormatVersion: 2010-09-09 Description: Automated S3 File Gateway S3 Glacier Archive Object Restores Metadata: License: Description: >- 'Copyright 2017 Amazon.com, Inc. and its affiliates. All Rights Reserved. Licensed under the Amazon Software License (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/asl/ or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.' Parameters: RestoreDaysValue: Description: Please enter number of days the object will be available after restore Type: String Default: 2 MinLength: '1' MaxLength: '3' AllowedPattern: '[0-9][0-9]*' ConstraintDescription: RestoreDaysValue must be a valid Integer! RestoreSpeedforGlacierArchive: Type: String Description: Please Choose how fast objects in Glacier/Archive should be restored AllowedValues: - Expedited # 1 to 5 minutes - Standard # 3 to 5 hours - Bulk # 5 to 12 hours RestoreSpeedForDeepArchive: Type: String Description: Please Choose how fast objects in Deep Archive should be restored AllowedValues: - Standard # 3 to 5 hours - Bulk # 5 to 12 hours Resources: FgwRestoreLambdaiamrole: Type: 'AWS::IAM::Role' Properties: ManagedPolicyArns: - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' Policies: - PolicyName: FgwRestoreLambdaiamRolePolicy0 PolicyDocument: Statement: - Action: - 's3:RestoreObject' - 's3:GetObject' # Required to perform headobject to track restore status - 's3:GetObjectVersion' # Required to perform headobject to track restore status Resource: '*' Effect: Allow AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: 'sts:AssumeRole' FgwRestorefunction: DependsOn: - FgwRestoreLambdaiamrole Type: 'AWS::Lambda::Function' Properties: Environment: Variables: ArchiveRecallTier: !Ref RestoreSpeedforGlacierArchive DeepArchiveRecallTier: !Ref RestoreSpeedForDeepArchive RestoreDays: !Ref RestoreDaysValue Runtime: python3.8 Timeout: 900 Description: Automated S3 File Gateway Archive Restores Function MemorySize: 128 Handler: index.lambda_handler Role: !GetAtt - FgwRestoreLambdaiamrole - Arn Code: ZipFile: | import os import logging import json import boto3 import base64 import gzip from botocore.exceptions import ClientError import io # Enable boto3 debug output #boto3.set_stream_logger("") # Set up logging # logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(asctime)s: %(message)s') logger = logging.getLogger(__name__) logger.setLevel('INFO') # Create S3 Service Resource: s3 = boto3.resource('s3') # Define Environment Variables: # Ensure Restore days value is integer and valid try: restore_days = int(os.environ['RestoreDays']) except ValueError: my_restore_days = 2 archive_restore_tier = str(os.environ['ArchiveRecallTier']) deep_archive_restore_tier = str(os.environ['DeepArchiveRecallTier']) # Ensure that the values set for Restore Tiers are valid and set a default value if archive_restore_tier not in ('Expedited','Standard','Bulk'): archive_restore_tier = 'Expedited' if deep_archive_restore_tier not in ('Standard', 'Bulk'): deep_archive_restore_tier = 'Standard' def lambda_handler(event, context): cw_data = str(event['awslogs']['data']) # Unzip and Unpack the Data cw_logs = gzip.GzipFile(fileobj=io.BytesIO(base64.b64decode(cw_data))).read() log_events = json.loads(cw_logs) for log_entry in log_events['logEvents']: result = process_recall(log_entry) logger.info(result) return { 'statusCode': 200, 'body': result } def process_recall(log_entry): logger.info("message contents: " + log_entry['message']) message_json = json.loads(log_entry['message']) if 'type' in message_json: logger.info("Found ErrorType") error_type = message_json['type'] logger.info("ErrorType = " + error_type) if message_json['type'] != "InaccessibleStorageClass": return "Unexpected error: not related to storage class" else: return_error = "error: no type entry" return return_error if 'bucket' in message_json: logger.info("Found Bucket") s3_bucket = message_json['bucket'] logger.info("Bucket = " + s3_bucket) else: return_error = "error: no bucket" return return_error if 'key' in message_json: logger.info("Found Key") s3_key = message_json['key'] logger.info("Key = " + s3_key) else: return_error = "error: no key" return return_error # Create S3 Object Resource s3_object = s3.Object(s3_bucket, s3_key) try: obj_restore_status = s3_object.restore obj_storage_class = s3_object.storage_class obj_archive_status = s3_object.archive_status if obj_restore_status is None: logger.info('Submitting restoration request: %s' % s3_key) # Add support for S3-INT storage class, restore for this class does not include restore days # Use the user defined restore tier for Glacier and S3-INT Archive_access otherwise default to Standard if obj_storage_class == 'GLACIER': result = s3_object.restore_object( RestoreRequest={'Days': restore_days, 'GlacierJobParameters': {'Tier': archive_restore_tier}}) elif obj_storage_class == 'DEEP_ARCHIVE': result = s3_object.restore_object( RestoreRequest={'Days': restore_days, 'GlacierJobParameters': {'Tier': deep_archive_restore_tier}}) elif obj_storage_class == 'INTELLIGENT_TIERING' and obj_archive_status == 'ARCHIVE_ACCESS': result = s3_object.restore_object( RestoreRequest={'GlacierJobParameters': {'Tier': archive_restore_tier}}) elif obj_storage_class == 'INTELLIGENT_TIERING' and obj_archive_status == 'DEEP_ARCHIVE_ACCESS': result = s3_object.restore_object( RestoreRequest={'GlacierJobParameters': {'Tier': deep_archive_restore_tier}}) else: restore_message = "Restore request already submitted!" return restore_message except ClientError as e: return_error = "Unexpected Error whilst attempting to recall object" logger.error(e) return return_error return result Outputs: RestoreDays: Description: number of days the object will be available after restore Value: !Sub ${RestoreDaysValue} GlacierArchiveTier: Description: How fast objects in Glacier/Archive will be restored Value: !Sub ${RestoreSpeedforGlacierArchive} DeepArchiveTier: Description: How fast objects in Deep Archive will be restored Value: !Sub ${RestoreSpeedForDeepArchive}