Cheat Sheet
Useful AWS tricks.
complete -C '$(which aws_completer)' aws
Make sure your IAM user has the following policy attached to it.
AmazonEC2ContainerRegistryFullAccess
Go to your AWS console, Services, Elastic Container Service.
Click on Create Repository.
Input your repository name and click on Next Step. AWS console will show pretty much the same instructions bellow.
Export your AWS credentials.
To retieve AWS ECR login command, run:
aws ecr get-login --no-include-email --region $AWS_DEFAULT_REGION
The return will be something like this:
docker login -u AWS -p eyWwBXlsg2FkIjoiL2hreVdiRWloWEM0NHFGcStaQ0hmYVZQOGJCVFFwODZPYXRrekZyaUZXY21pRVZNL09qMHMzRFM2S2Y5MkFleGJhYUxjeW1GR2srTFc5VUYxejBVWFFpakVuclhqM2JJeWN6MDh2d3pGZVE1YXA0NVYrSnArRDZCNkdpQWhuK3AycUhmdEpKTU1yM2srNFZ5OWdheC85UHN5WmhVSDJZUklwRC9tYTQ2MWdLNUVjNnpycWovMytOY2V2RTNkamxNTWg1MXBCMkpMVG9TM3prWUhacDN2b1IzazBDM25qV2thVlNDYy9NbnFxTTJjaHdvQWEwbG1pUnU0ZGVUN1BRWXhZUUVUb1VsNjZkR1RZcGdtZ2RBbWh0TXZTbjcwWVN1c2djVEhtV3dLVGF3VjlRcjh1d21Ybm0vbExqL2VmZm04NllXeUxBaS9zb3g5akhIMHZsT2tydlZ5bEtUT3VySFNxQWdFamRCSndQM1F4U1gzd1lBc0x6Uzk0andiTW41VWwxTXYzNzZDVkxZc09lRVBjM1UxM0tuMFhSMGt3Y0V1L282SWlyc1BUSGh5Tnc1THF0S0swLy8vWm5SSWlTWFhhaWEvbU50c3J6OWhYRWFqV1VRVjQ4M09sbGpVT0JnUzJhdm9MSkkyU2tVSjRiMzBrdnVEOGNJVjVTajk3QzJRNzk4Q2ppUWVzV2JuV2RMN29Zd1JPbHc2dVViTlhQb244V1QwWDdPVjQyandmSXFuL1A3ODJad2p3a2QrVFVBNTRYc1UrQ1dPcmF6VG9oMDZ3OWczVmVZZnJJc0RaOHNIYnVZYkoxaDZwV1VsU2RZYURDTXJwRE9SUEN5YUpBWFFsVlZ1TWJJR2ZacXQ2bGRORXdFR0NVd1QrUWZpSzNuR0RCNkZVREx3Q0FEcGJoVEdjRllpWDIyVENVd0JHUkc5aGFSaU00NXV6V3pVLytJNk9TOFZPdmlSdTA5WHQxTms2cmNEeHJ4OE5DR0pNTGJkdGFLSVI3MW9aY1BYNS9JejJDdDZzYUZvekczVTQrYjZFV1ErbnRUeFZGakxRWmxwOTN1TUpVc3VyTDFQK3Y0MnBsSnNENXhTeFhHdy80aFhjVUlzOElQQkRmRjZEYWJxNC8vZCtMOVRKS2ZJakpwRXdrc1A3UVp3UVJWdURPM1lGV2NiakhvMmxUMXkwQ3RIUWNXZkRVeS9uMitPZHZIWmd1b0NSaU45elRoa0haRE9lcmtFU3FIR3hwK2x2eXpSMnR6NSs0VXhjQTJlR1FJaTFuVnB6MWFnbzY1cTM1WW1JREFIOUMvUDN2WmVZbVRpQTIxRW1tVnZaMURCK1hVbVlwUURkbGpOMW82SUZRbktzWGRlU0FFMTFsTUNxVlFnTXJUS25BbWhDTzIvMStBLzNKVmNEbC9pY1FCWGlvUFhXVmRlNU1WK2VnN3ViSWJyVGw0clVRdkt0VlFXdWVSdmVYWUhMeTlTa2lqWXhZSkZHTU53cnFMeERpWFhxOGIrQjZLVlFrcnVnPT0iLCJkYXRha2V5IjoiQVFFQkFIaEx5ano3UEpFUG1ZYnFINkE2Zi93V056eXo1QkthMzdCbkJQc0F5dUYxS1FBQUFINHdmQVlKS29aSWh2Y05BUWNHb0c4d2JRSUJBREJvQmdrcWhraUc5dzBCQndFd0hnWUpZSVpJQVdVREJBRXVNQkVFRFA5SWFOV0FBTFgvd3pBYUlnSUJFSUE3VlFiY3NkU3Z4UjMzZ1I0WDRPbldQTnBScFRhTythdFRWajFPQXErNFpjOWM0eGhqeTJ2UmtjaFJIcmdxUERycmI4OFZQZmNITEgrUkxCZz0iLCJ2ZXJzaW9uIjoiMiIsInR5cGUiOiJEQVRBX0tFWSIsImV4cGlyYXRpb24iOjE1Mjg0MzU3MTB9 https://189013708713.dkr.ecr.ap-southeast-2.amazonaws.com
Copy and run your login command.
Tag your Docker image (replace with your info):
docker tag \
YOUR-IMAGE-NAME:latest \
189013708713.dkr.ecr.ap-southeast-2.amazonaws.com/YOUR-IMAGE-NAME:latest
Then push it to AWS ECR.
docker push \
189013708713.dkr.ecr.ap-southeast-2.amazonaws.com/YOUR-IMAGE-NAME:latest
Create a Security Group that will be used for your lambda function.
Go to your lambda function setup on the AWS console.
Put the Lambda function in the same RDS VPC.
Add the Lambda function in all subnets.
Add the Security Group you just created.
In your RDS security group, allow your lambda function security group.
Go to Services -> VPC -> Endpoints -> Create Endpoint.
Select AWS Services as the Service category, and then, in the Service Name list, select the Secrets Manager endpoint service named com.amazonaws.us-west-2.secretsmanager
Specify the VPC you want to create the endpoint in. For this post, I chose the VPC named vpc-5ad42b3c where my RDS instance and application are running.
To create a VPC endpoint, you need to specify the private IP address range in which the endpoint will be accessible. To do this, select the subnet for each Availability Zone (AZ). This restricts the VPC endpoint to the private IP address range specific to each AZ and also creates an AZ-specific VPC endpoint. Specifying more than one subnet-AZ combination helps improve fault tolerance and make the endpoint accessible from a different AZ in case of an AZ failure. Here, I specify subnet IDs for availability zones us-west-2a, us-west-2b, and us-west-2c:
Select the Enable Private DNS Name checkbox for the VPC endpoint. Private DNS resolves the standard Secrets Manager DNS hostname https://secretsmanager..amazonaws.com. to the private IP addresses associated with the VPC endpoint specific DNS hostname. As a result, you can access the Secrets Manager VPC Endpoint via the AWS Command Line Interface (AWS CLI) or AWS SDKs without making any code or configuration changes to update the Secrets Manager endpoint URL.
Associate a security group with this endpoint. The security group enables you to control the traffic to the endpoint from resources in your VPC. For this post, I chose to associate the security group named sg-07e4197d that I created earlier. This security group has been set up to allow all instances running within VPC vpc-5ad42b3c to access the Secrets Manager VPC endpoint. Select Create endpoint to finish creating the endpoint.
To view the details of the endpoint you created, select the link on the console.
The Details tab shows all the DNS hostnames generated while creating the Amazon VPC endpoint that can be used to connect to Secrets Manager. I can now use the standard endpoint secretsmanager.us-west-2.amazonaws.com or one of the VPC-specific endpoints to connect to Secrets Manager within vpc-5ad42b3c where my RDS instance and application also resides.
Now that I have created the VPC endpoint, all traffic between my application running on an EC2 instance hosted within VPC named vpc-5ad42b3c and Secrets Manager will be within VPC. This connection will use the VPC endpoint and I can use it to retrieve my RDS database secret stored in Secrets Manager. I can retrieve the secret via the AWS SDK or CLI. As an example, I can use the CLI command shown below to retrieve the current version of my RDS database secret:
aws secretsmanager \
get-secret-value \
–secret-id \
MyDatabaseSecret \
–version-stage AWSCURRENT
Since my AWS CLI is configured for us-west-2 region, it uses the standard Secrets Manager endpoint URL https://secretsmanager.us-west-2.amazonaws.com. This standard endpoint automatically routes to the VPC endpoint since I enabled support for Private DNS hostname while creating the VPC endpoint. The above command will result in the following output:
{
"ARN": "arn:aws:secretsmanager:us-west-2:123456789012:secret:MyDatabaseSecret-a1b2c3",
"Name": "MyDatabaseSecret",
"VersionId": "EXAMPLE1-90ab-cdef-fedc-ba987EXAMPLE",
"SecretString": "{\n \"username\":\"david\",\n \"password\":\"BnQw&XDWgaEeT9XGTT29\"\n}\n",
"VersionStages": [
"AWSCURRENT"
],
"CreatedDate": 1523477145.713
}
Go to AWS Secrets Manager -> Secrets -> YOUR SECRET
Get the examples in "Sample code" and add to your lambda function code.
aws s3 mb s3://BUCKET-NAME
aws s3 ls s3://YOUR-BUCKET-NAME
aws s3 ls \
--summarize \
--human-readable \
--recursive \
s3://BUCKET-NAME
aws s3 cp \
--recursive \
s3://BUCKET-NAME \
DESTINATION-FOLDER
aws s3 cp \
--recursive \
FILE-OR-FOLDER \
s3://BUCKET-NAME
aws s3 sync \
LOCAL-FOLDER \
s3://BUCKET-NAME
Head over to EC2 –> Volumes and create a new volume of your preferred size and type.
Select the created volume, right click and select the "attach volume" option.
Select the instance from the instance text box as shown below.
Now, login to your ec2 instance and list the available disks using the following command.
lsblk
Example output:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 91M 1 loop /snap/core/6405
loop1 7:1 0 12.7M 1 loop /snap/amazon-ssm-agent/495
loop2 7:2 0 17.9M 1 loop /snap/amazon-ssm-agent/1068
loop3 7:3 0 18M 1 loop /snap/amazon-ssm-agent/930
loop4 7:4 0 91.1M 1 loop /snap/core/6259
loop5 7:5 0 91M 1 loop /snap/core/6350
xvda 202:0 0 300G 0 disk
└─xvda1 202:1 0 300G 0 part /
xvdf 202:80 0 1000G 0 disk
In this case, the device name is
xvdfMake sure the volume has no data:
file -s /dev/xvdf
Example output:
/dev/xvdf: data
Format the disk:
mkfs.ext4 /dev/xvdf
Label the disk:
e2label /dev/xvdf "storage-001"
Mount the disk:
mkdir -p /storage/storage-001
mount /dev/xvdf /storage/storage-001
Configure fstab:
cp /etc/fstab /etc/fstab-BKP
nano /etc/fstab
Add:
LABEL=storage-001 /storage/storage-001 ext4 defaults,discard 0 0
Test fstab:
umount /storage/storage-001
mount -a
df -h
Create a snapshot of the volume you want to increase size.
Create a new volume from the snapshot.
Attach the new volume.
SSH to the instance an run:
e2fsck -f /dev/xvdh
resize2fs /dev/xvdh
Umount current volume:
umount /data
Edit fstab, comment out the old volume and add the new one:
#/dev/xvdg /data ext4 defaults 0 0
/dev/xvdh /data ext4 defaults 0 0
Mount it:
mount -a
Last modified 3yr ago