AWSome Day Notes

This post represents my personal rough notes from the AWSome Day Conference held at Hotel Football in Manchester on 28th February 2018.

Key note - Ric Harvey - @ric__harvey

• Twitter hashtags: #awsomeday & #aws
• VPC Link (since last November) - allows 3rd party vendors to drop a server into your AWS account but they manage it for you.
• AWS has the largest number of start-up customers (born in the cloud) AND the largest number of enterprise customers too along with the broadest ecosystem of ISVs.
• Just announced Amazon MQ - a more "standard" message queue as an alternative to Amazon SQS.
• Simon Wardley - big speaker on the circuit (mostly in London) who is a strong advocate of serverless computing.
• Can provision a VMWare vSphere suite in AWS on bare metal!
• Amazon released 24 improvements to AWS in 2008, and 1300 in 2017!
• AWS now has a graph database called Neptune.
• AWS Fargate allows managing container clusters without worrying about the underlying infrastructure.
• Amazon SageMaker is a new tool for machine learning. You can easily build, train, and deploy machine learning models.

Tom Woodyer - @AWSTomWoodyer (woodyert@amazon.com)

• Tom shows a demo of Amazon Rekognition which analyses photographs to determine tags for the photo. Does object recognition, facial recognition, sentiment analysis etc.
• The guys at Netflix that created Chaos Monkey now have their own company that offers "failure as a service" called Gremlin.
• The Amazon 3 day advanced architecture course goes into building a 800.88 certified secure network.
• All AWS services map back to the "AWS Global Infrastructure" (this is the regions and availability zones across the world - data centres!)
• Tom says that the very first AWS service was SQS!
• Tom says that the entire day's sessions in the technical track are 100-level talks! :(
• Tom shares the fact that around 5% of all businesses IT is wasted - it's completely unused.
• Traditional terminology to AWS terminology: Firewalls, ACLS, Admins => Security Groups, Network ACLs, AWS IAM (Security), Router, Network pipeline, switch => ELB, VPC (Networking), On-Premise servers => AMI, EC2 (Servers), DAS, SAN, NAS, RDBMS => EBS, EFS, RDS (Storage and databases).
• AWS now has 18 regions. Always at least 2 availability zones per region.
• AZ's are linked within a region with 180TB networking. Sub-10ms network!
• AWS has a concept of the "well-architected" framework which is the combination of security, reliability, performance efficiency, cost optimization, operational excellence.
• AWS Trusted Advisor helps to improve each of the above pillars of "well-architected". Can show cost optimization of low utilisation services etc., fault tolerance recommendations - i.e. running services in only a single AZ.

Amazon Elastic Cloud Compute (EC2) - Tom Woodyer

• Use Tags!!! Whereas Azure has resource groups that group resource into a single entity (so that all resources can be deleted all together), AWS doesn't have resource groups, but uses tags for this purpose. Resources in AWS can have up to 50 custom tags and then you can run a report to show all resources with a given tag that has a given value!
• It's important to know how long launching an EC2 instance (can be different between Linux / windows etc. or if the custom AMI/OS has complex setup/start-up scripts) will take as this has to be taken into account when adding auto-scaling health checks. You need to "delay" the health check to give the instance sufficient time to spin up and you need to know this time yourself.
• AWS Marketplace AMIs contain both AMI that include a software cost as well as the EC2 instance cost. Some AMIs have zero software cost as they are BYOL (Bring your own license) versions so you have to supply your own license.
• Can perform a "vertical resize" of an EC2 instance by shutting it down and then altering the size of the instance!
• Recommended approach to creating a custom AMI is to create running instance, configure it all, shut it down, create a snapshot (EBS snapshot), then from the EBS snapshot, create an AMI image.

Amazon Simple Storage Service (S3) - Tom Woodyer

• You can configure access to S3 to be limited to HTTPS rather than HTTP. This allows you to ensure compliance with things like PCI but its on you to do it.
• Currently S3 has over 25 trillion objects stored within it!!!!
• When the Singapore region opened, amazon delivered more than a ton of SSD Drives per day to the data centre.
• S3 supports 7 different ways to encrypt data inside it. You can store keys in AWS KMS and they'll encrypt the data for you using your KMS key as well as other server-side key management options.

Security, Identity & Access Management (IAM Services) - Tom Woodyer

• AWS has a shared responsibility model. Amazon (and only Amazon) is responsible for AWS global infrastructure (AZs, regions, edge locations) and AWS foundation services (compute, storage, database, networking). Users are responsible for everything else (client/server side encryption, network traffic protection, OS, platform, applications, identity & access management - customer applications and content)
• AWS is responsible for security OF the cloud. Users are responsible for security IN the cloud.
• AWS has 58 compliance frameworks that they adhere to in the areas where it matters.
• AWS IAM provides federated access so on-premise active directory can be automatically replicated in AWS.
• All AWS permissions are explicit. They must be explicitly given to a user or group. Nothing is provided "by default" or given implicitly.
• Can set a user's permission to read only access then allow the "assume to be admin" permission which allows access to STS (secure token service) and allows the user to elevate their permissions from read-only user to admin.
• Can configure access from a user in one AWS account to another user in a different AWS account! (uses the same federated access).
• Can add conditions to such federated access. i.e. must be a user who has supplied an MFA (multi-factor authentication) code.
• AWS IAM Best practices: Delete AWS account (root) access keys, create individual IAM users, use groups to assign permissions to IAM users, grant least privilege, configure a strong password policy, enable MFA for privileged users, use roles for applications that run on EC2 instances, delegate by using roles and more.
• Tom talks about how the ticketing industry is "shark infested". Some denial of service attacks involve other ticketing providers starting a ticket purchase such that the ticket is "held" for some amount of time for the user to complete, but purposefully abandon the purchase in order to prevent real users from buying the ticket!
• AWS experiences attacks 24/7/365 but they successfully deflect them and can determine (to a large extent) which traffic is legitimate (and so is allowed into the network) versus which is malicious (and so is blocked at the edge).

RDS and DynamoDB Services - Tom Woodyer

• AWS Managed Database Services - DynamoDB - noSQL database, ElastiCache - in-memory cache based on MemCacheD or Redis, RDS - supports SQL Server, Oracle, MySQL, MariaDB, Aurora, RedShift - petabyte-scale data warehouse, AWS database migration service - allows moving on-premise DB's (i.e. Oracle, SQL Server etc.) to the cloud with no down time!
• Nowadays all AWS RDS databases are isolated database services so run inside the VPC on your account.
• RDS backups are enabled by default, retention up to 35 days. Manual snapshots can be taken manually and stored in S3.
• Oracle and SQL Server in AWS RDS can still support TDE (transparent data encryption).
• RDS's can be configured for "multi-AZ" mode. Has a redundant fail-over RDS instance in another AZ (but this does cost more).
• Single best RDS best practice is to test fail-over of RDS instance.
• RPO & RTO. Recovery point objective (how much data can you afford to lose). Recovery time objective (how long does it take to recover to the RPO point) - DR terms.
• DynamoDB tables need items with a partition key which must be unique or a partition key and sort key which must be (in combination) unique. Partition keys will often determine the physical location that AWS will use to store the data so reading multiple items with the same partition key is optimized.
• Aurora gives up to 10x the performance of MySQL DBs and up to 3x the performance of PostgreSQL DBs.

Elasticity & Management Tools - Tom Woodyer

• Elasticity is achieved with a triad of services, ELB (Elastic Load Balancing, Auto-Scaling and CloudWatch).
• ELB notifies CloudWatch of latency and utilisation and when it hits the configured metrics (determined by the user), it notifies auto-scaling to spin up further instances.
• Amazon.co.uk - minimum number of servers 300. Tom has seen up to 900 servers!
• AWS has 3 different load balancers - Classic load balancer, Application Load Balancer (ALB) and Network Load Balancer (NLB). NLB redirects traffic at IP level.
• You register instances in a target group and direct traffic to a target group. You can have a group with a single server that doesn't scale beyond one server, but it can restart that instance if something happens to it!
• Load balancers have "stickiness". If the server is still up, AWS load balancing will continue to direct the same user to the same server.
• CloudWatch allows installing a component on EC2 instances which will push application level metrics to CloudWatch. It can even run on an on-premise server.
• AWS accounts have a soft limit of 20 EC2 instances per region. This is to prevent auto-scaling with a maximum set to 0 (zero) (which actually means NO MAXIMUM!) and auto-scaling goes mad an creates 1000 instances!
• Auto-scaling can be triggered now only by settings that can be set in the web console, but also by an API endpoint call. Tom mentions how TicketMaster had scaling triggered by an API call called by client-side JavaScript from MouseOver events!!!!
• Amazon receives 180 million CloudWatch metrics per second on a global basis!