Amazon QLDB is a fascinating new service from AWS that sits in the database category. This article looks at an automated way of creating any required tables and indexes.

Getting Started

The first step with QLDB is to create a ledger, which represents the database instance. From an application perspective, when you interact with QLDB, you interact with tables you have created using PartiQL (a SQL-compatible language). These tables store revisions of documents, in an Amazon ION format.

QLDB can be considered a schemaless database, as there is no schema enforced on the data in any table. Instead, any schema must be enforced by the application once the data has been read. QLDB is also considered a serverless database. When you create a ledger, you do not specify any DB instance size. You also don’t specify any read or write capacity units. Importantly, you are not even provided with a username or password to log into any database. This provides one of the guarantees around immutability. You can guarantee that nobody with administror rights has amended any data, as there is no capability to do so.

Ledger, Table and Indexes

You can create a new Ledger in QLDB using CloudFormation or the CLI, as well as through the administration console. The next challenge comes in creating a table with an index.

To insert data into QLDB you need a table. It is also strongly recommended that you add relevant indexes to a table. This is important as it prevents SELECT queries from doing a full table scan, so can significantly improve performance. There are restrictions with the use of indexes as set out below:

• They can only be created on empty tables
• They can only be created on a single field. Composite indexes and range indexes are not supported.
• They cannot be dropped once created
• There is a maximum of 5 indexes per table
• Query performance is only improved when you use an equality predicate e.g. fieldName = XYZ

This means that thought needs to go into the database design upfront.

One problem is there is no way to create a table or index using built-in QLDB CloudFormation, as the only supported resource type is that of a Ledger. This is unlike other database services such as DynamoDB, which has CloudFormation support for creating a table, along with associated properties such as the partition key and sort key, and local or global secondary indexes.

However, there is still a way to achieve this through the use of custom resources.

Custom Resources

Custom resources enable you to write custom provisioning logic in templates that AWS CloudFormation runs anytime you create, update (if you changed the custom resource), or delete stacks.

The following is a snippet from a serverless.yml file to show how this is achieved:

resources:
  Resources:
    qldbGuideLedger:
      Type: AWS::QLDB::Ledger
      Properties:
        Name: qldb-simple-demo-${self:provider.stage}
        DeletionProtection: false
        PermissionsMode: ALLOW_ALL
        Tags:
          - 
            Key: name
            Value: qldb-simple-demo

    qldbTable:
      Type: Custom::qldbTable
      DependsOn: qldbGuideLedger
      Properties:
        ServiceToken: !GetAtt CreateTableLambdaFunction.Arn
        Version: 1.0  #change this to force redeploy

    qldbIndex:
      Type: Custom::qldbIndexes
      DependsOn: qldbTable
      Properties:
        ServiceToken: !GetAtt CreateIndexLambdaFunction.Arn
        Version: 1.0  #change this to force redeploy

This shows how the custom Lambda function to create the index is only invoked once the Lambda function to create the table has successfully run, and this in turn is dependent on the creation of the ledger itself.

The ServiceToken is the ARN of the function that CloudFormation invokes when you create, update or delete the stack. The name of CreateTableLamdaFunction.ARN is the Logical ID in the CloudFormation that is created by the Serverless Framework for a function defined as createTable in the functions section.

The full working example can be found in QLDB Simple Demo and has been tagged using v0.2

Note that the simple demo only implements the Create request type, and returns a success for other request types. This is because the custom resources are part of the same CloudFormation stack, and deleting the ledger will automatically result in the enclosed tables and indexes being deleted.

Useful Links

For more details, check out Amazon QLDB Guide and you can find a curated list of QLDB guides, development tools and resources at Awesome-QLDB

• Ledger
• QLDB