Background

Have you ever wanted the power of the AWS CodePipeline mixed with the convenience of the AWS CDK pipeline and supporting services in a CDK app?

Background

At Virtuability, we’re always on the lookout for tools that can enhance our AWS consulting services and deliver maximum value to our customers.

Background

Over the years, we have helped organisations make sense of their CloudTrail logs when the need arises to query for specific events or to aggregate and report on the data. Our tool of choice is generally Amazon Athena to accomplish this.

I start this post by saying I’m not a professional software developer, I work mainly in IT Operations, although I write especially for IAC and small lambdas functions.

When developing a Lambda function most of the time I need to interact with AWS Services via the famous boto3 library; boto3 is a powerful library developed and maintained by AWS which provides a communication framework to interact with native AWS Cloud Services.

Background

As organisations continue to grow and with the resulting increases in AWS consumption there is a need to be able to cost-effectively scale not just workloads but also governance and guardrails to ensure that security requirements are met across the estate.

In a fast-paced cloud environment governance is fundamental, defining standard policies for deployment and shifting left controls are key to successful organisations. Thanks to AWS Landing Zone and AWS Guardrails organisations can confidently deploy, control, and audit their resources and developments.

Have you ever created a Network Load Balancer on AWS, its target group and its target, and some security rules attached to it, to end up giving out to your browser because you couldn’t reach the target?

How often are you deploying a Lambda container image, basically a Lambda running on a Docker image, for a platform that doesn’t match your localhost platform? Often I deploy functions running on ARM rather than on X86_64, this is a personal preference and it doesn’t come with any massive advantage (there are online some comparison), and although my laptop is ARM-based, the CI/CD server is not :/

I’ve been working with CDK and I think is brilliant, the way it lets you define resources and infrastructure using your favorite coding language is awesome, I personally use Python. Sometimes what happens is that CDK takes over a lot of control and creates resources as it thinks is proper… Also, documentation lacks some advanced configuration.

With AWS IAM Identity Center, formerly known as AWS Single Sign-On, it became simpler to integrate identity providers such as Azure AD, JumpCloud etc across the whole AWS organization.

Cloudformation support in turn enabled simpler and more consistent, declarative provisioning of Permission Sets in the Organization.

Background

In November 2020 AWS announced that Security Hub now integrates with AWS Organizations.

Unlike for many other AWS Organizations services integrations you will not find the ability to enable Security Hub on the Organizations page in the Master account.

Background

With the advent of the Raspberry Pi 4, Pi’s are sufficiently powerful in terms of both CPU and memory for AWS development. Furthermore, AWS has recently made significant headway in the ARM space with the release of Graviton-based EC2 and support for ARM 64-bit (aarch64) with the following services:

Background

With the advent of the Raspberry Pi 4, Pi’s have become quite powerful both in CPU and memory terms and are now good candidates for software development on ARM architecture. The Pi 4 boasts quad-core ARM v8 1.5Ghz CPU with 64-bit support and the option of 2, 4 or 8GB of DDR4 memory. It also has excellent connectivity through dual HDMI support (including 4k), Gigabit Ethernet and USB 3.0 (and 2.0) ports.

Background

It’s much faster to be able to develop and debug AWS Glue / PySpark scripts locally.

The Developing and Testing ETL Scripts Locally Using the AWS Glue ETL Library instructions describe installation but are not complete. There are certain dependencies to consider to make this work.

Introduction

Using Serverless combined with Cognito can be a great way to eliminate the real estate as well as development and operational footprint when it comes to authentication and authorisation stacks.

Introduction

If correctly composed, logs can be an extremely useful resource to tap into in the following use cases:

• Support end-users
• Derive business metrics (how many users used our service yesterday, over the last 7 days and in the past month?)
• Derive operational metrics (service uptime and failures)
• Feed metrics to generate alerts during abnormal events or to trigger capacity increases and decreases based on service loads
• Find and fix bugs

Business & Operational Dashboards can today be built to aggregate and chart metrics derived from logs in near-realtime. Dashboards can be tailored not only to IT but also to the wider organisation.

Introduction

Why adopt DevOps?

IT change can be painful and subject to long lead times in many organisations. The pain generally stems from treating change as exceptional rather than business-as-usual - often in the form of running a project to effect the change.

Background

We have recently completed a Serverless & DevOps transformation project with one of our clients, CitizenMe. CitizenMe presently has more than 200.000 global end-users and has processed millions of transactions since inception.

In military terms a Landing Zone is an area where aircraft can land; in effect a base camp from where operations can extend.

AWS has for the last year or two used the term Landing Zone to convey an infrastructure foundation and security baseline on which applications and services can “land”. The applications inherit & adopt a set of shared services, integration and design patterns. The purpose of the Landing Zone is to establish an organisational baseline that supports its requirements for infrastructure and security and is rooted in “best practices”, which seek to balance business and security risks against innovation and value.

AWS Cross-Account Roles are an excellent way of managing access to a target account (the account in which work is carried out) from other AWS accounts. Some scenarios to consider in this context include:

For both test and build purposes I often find myself reusing parts of past CloudFormation templates. Over time I’ve found that the foundation of the templates like VPC, subnets, routing tables etc remain roughly the same. I have also found that the AWS VPN solution often isn’t suitable because of e.g. NAT, lack of port forwarding, lack of “hardware VPN”, expertise etc.

AWS CloudFormation size limits are well-documented in the User Guide. However, this does not make hitting any of the limits any less painful. I recently hit the template body size limit in request (–template-body) of 51200 bytes on one of my templates. This adds the extra complexity of having to first upload the template to an S3 bucket. When you’re quickly iterating changes this becomes rather onerous very quickly.

Over the years I have witnessed, proposed and implemented a wide range of AWS use cases; and few of them actually belong in the sexier cutting-edge, containerised, hyper/auto-scalable, serverless micro-services realm. I mostly find a certain level of pragmatism - rooted in both tactical and strategic choices - involved in the adoption of Cloud:

One of the largest concerns of allowing AWS API calls to be made from the outside is issuing an API key and secret for developer and administrator PCs and laptops alike because they may be interceptable in one way or another. Some scenarios spring to mind: