July 7, 2019
Interesting Articles This page links to interesting articles grouped by topic
Serverless Yan Cui - Blog Site Chaos Engineering Norah Jones - Chaos Engineering Traps Basiri et al - Automating chaos experiments in production Daniel Bryant - How Did Things Go Right? Summary Tracing Cindy Sridharan - Distributed Tracing — we’ve been doing it wrong Observability Epsagon - Serverless Observability Workshop
April 1, 2021
Background Concurrency control is critical in any system where accuracy and trust in the record needs to be maintained. It is the way of ensuring that correct results are generated, even when multiple concurrent transactions are taking place.
This blog post takes a closer look at how Amazon QLDB implements concurrency control. It includes a number of demos that you can try out for yourself, with more information and the code available at QLDB Concurrency Demo
August 4, 2020
Background In a previous post I described how to stream data from QLDB to DynamoDB to support single-digit latency and infinitely scalable key-value enquiries.
Another common use case is to support rich text search and downstream analytics such as aggregation or metrics across records. For this, I look to build on the previous post and stream data from QLDB into Amazon Elasticsearch (ES). To try this out for yourself, the code is available on GitHub - QLDB Simple Demo.
July 23, 2020
Introduction I’ve recently been building out a serverless application on AWS that interacts with Amazon QLDB as a purpose-built database at the backend. In many cases, making a few simple configuration changes can have a dramatic impact on performance. This post looks at some free tools and services that you can use to help optimise your own serverless application. For demonstration purposes, I focus on QLDB but also detail a brief comparison with DynamoDB.
July 5, 2020
Background In September 2019, AWS announced the general availability of QLDB, a fully managed centralised ledger database. At the heart of QLDB is its append-only journal. All requests go through the journal first, and it only contains committed transactions. This data is immutable, and creates a complete audit trail of every single change ever made - essential in a digital world that needs trust in the data, and must meet regulatory compliance.
June 27, 2020
Background In a previous blog post I wrote about the significant improvements made in the Amazon QLDB driver for Nodejs. At the time it had gone to 1.0.0-rc.1, and following some more updates, it has now gone to v1.0.0.
Significant Updates In this period, there were three important updates. For more details check out the release notes
Session pooling functionality moved to QldbDriver
In previous versions, there was both a QldbDriver and a PooledQldbDriver with strong recommendations to use the PooledQldbDriver to enable a pool of sessions maintained for you that reuse underlying connections.
June 14, 2020
After building out some simple backend examples using Amazon QLDB, it’s now time to build out a full stack application to really bring a combination of AWS services to life and show whats possible. Luckily, to do that I’ve been joined by Chris Williams and Greg Simons and we’ve set up a GitHub repository for anyone else interested in taking part.
The aim is that this will be a series of blog posts stepping through some of the process.
May 27, 2020
Taking the AWS SA Pro Exam First off, well done to anyone starting the journey to certification. Just being willing to study and take an exam puts you ahead of most others, so you should be rightly proud. For me, I like the goal of becoming certified and the structured content as a way to keep me motivated.
I was fortunate enough to pass the AWS SA Pro exam a few days ago with an online proctored exam.
April 4, 2020
Amazon QLDB Driver for NodeJS On Friday 3rd April, the Amazon QLDB team announced release v1.0.0 of the Amazon QLDB Driver for NodeJS. I was excited to give this a try, and see how it compared to the preview version. This is a brief summary of my initial findings.
Setting Up The driver has a number of peer dependencies:
February 8, 2020
Background In a previous blog post I wrote about creating a ledger with a table and index in Amazon QLDB. This post expands on that by providing a simple API to create a new vehicle record, and get an existing vehicle record. The code all exists as infrastructure as code in this Github Repo.
Serverless Framework The code uses the Serverless Framework to set up all the resources. The configuration for the create vehicle function is shown below:
February 2, 2020
Background Artificial Intelligence (AI) and Machine Learning (ML) are big news and certain to have a disruptive effect on organisations. Over the past few years, the likes of recommendation engines and voice assistants have become part of my everyday life, and its a topic I want to get closer to. The challenge was that it traditionally seemed to require a PhD and deep expertise in algorithms. However, AWS have started bringing AI and ML into the hands of developers, with the introduction of hardware devices like AWS DeepLens and AWS DeepComposer.
January 26, 2020
Amazon QLDB is a fascinating new service from AWS that sits in the database category. This article looks at an automated way of creating any required tables and indexes.
Getting Started The first step with QLDB is to create a ledger, which represents the database instance. From an application perspective, when you interact with QLDB, you interact with tables you have created using PartiQL (a SQL-compatible language). These tables store revisions of documents, in an Amazon ION format.
January 7, 2020
A few months ago, I was incredibly honoured to be recognised as an AWS Hero. Off the back of this, one of my goals for the coming year is to blog more and try and raise awareness of technologies I am interested in.
I always find its easier to learn, when you are passionate about a subject. Over the past few years I have thrown myself into serverless. You could see from all the major cloud providers that this was the direction the industry is quickly heading in.
September 28, 2019
Background Yesterday, Chris Munns updated a blog post stating that the Lambda VPC networking improvements have been fully rolled out to the US East (Ohio), EU (Frankfurt), and Asia Pacific (Tokyo) regions. This gave me a chance to run a simple test to see if I could notice any reduction in latency.
The Setup To setup the test, I created a quick project to deploy a lambda function associated with a VPC, and one that was not.
September 22, 2019
What is QLDB? Amazon QLDB is a fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log owned by a central trusted authority.
QLDB is called ‘Quantum’ as in indivisible discrete changes. All the transactions are recorded to a transparent journal where each block represents a discrete state change.
Overview QLDB is considered a ledger database. A ledger consists of one or more QLDB tables, and a journal, as shown below:
July 7, 2019
Securing Static Website In a previous blog, I wrote about setting up this website. It is a simple static site with a basic architecture as shown below:
The teachmyselfcloud domain is registered with Route 53. This automatically creates a hosted zone, with a NS record and a start of authority SOA record. I created an alias A record that points to the CloudFront distribution, that sits in front of the S3 bucket that holds the individual objects.
June 23, 2019
AWS Organizations Terminology and Concepts Organization An organization is the entity that you create to consolidate your AWS accounts
Root The root is the parent container that is automatically created when you create an organization. Currently, you can only have one root.
Organizational Unit (OU) An organizational unit is a container for accounts within a root. You can create a hierarchy by creating an OU within an existing OU. You can then add AWS accounts to an OU.
June 22, 2019
Identity and Access Management (IAM) AWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely. IAM is global, and this means when you create users, groups and roles, they will be accessible in any region.
User An IAM user is an entity that you create in AWS. The IAM user represents the person or services who uses the IAM user to interact with AWS.
June 22, 2019
AWS Virtual Private Cloud Amazon Virtual Private Cloud (Amazon VPC) lets you provision a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define. You have complete control over your virtual networking environment, including selection of your own IP address range, creation of subnets, and configuration of route tables and network gateways. You can use both IPv4 and IPv6 in your VPC for secure and easy access to resources and applications.
August 29, 2019
My development background started with C, before turning to Java, and that’s where it stayed. Over time, I realised this was a mistake. You’ve got programming languages that are statically typed, dynamic, compiled, interpreted, object-oriented, procedural, functional and everything in between. Each of them have advantages and disadvantages and use cases where they are most appropriate. Having knowledge of only one is like believing you can solve every problem with a hammer.
August 30, 2019
Tip 1: Packaging of aws-sdk The aws-sdk comes bundled with the Lambda execution environment, but you should not rely on this. Instead you should pack the aws-sdk in your deployment package because:
the one in the Lambda execution environment is always a few months old, so probably missing patches, security updates, etc. it’ll introduce difference to what I had tested AWS can update it without notice, so far I haven’t heard of anyone running into issues in Node.
June 9, 2019
Creating AWS Accounts Best practices when using AWS at an enterprise level are to use a multi-account strategy, which helps provide isolation, greater control and limited blast radius. This is highlighted in the AWS Landing Zone.
In order to play around with new services like AWS Organizations and fully understand cross-account access and implications, I have set up 3 separate AWS accounts
Set up AWS CLI I find it easier to use the AWS Command Line Interface (CLI) to call the APIs of AWS services directly.
June 8, 2019
I previously set up this blog with the awesome Hugo open source static-site generator. Last time I used GitLab Pages. I also recently setup a static site using GitHub and Netlify. However, I want to have another go at resurrecting this site, and use it to keep study notes for exams and personal projects. This time, I’ve decided to go all in on AWS.
Set up CodeCommit First I created a new repository on CodeCommit.