Performance

Performance is one of the tent poles of aws-lite. We take it seriously because we want your applications to be as fast possible.

As such, we regularly test and publish open, reproducible, real-world metrics for every key aspect of performance, comparing aws-lite to AWS’s own aws-sdk (v2) and @aws-sdk (v3). Learn more and view source at the aws-lite performance project on GitHub.

All metrics are published below (or skip straight to the wrap-up).

Methodology

In addition to publishing our source, raw data, and final results, we believe it’s important to share the details of our performance testing methodology, too. Learn more here.

Stats last updated: 2024-04-19T01:37:24.301Z

Coldstart latency

Coldstart latency measures the impact of each SDK on AWS Lambda coldstarts – the pre-initialization phase where your code payload is loaded into the Lambda micro VM.

In these stats we expect to see lower values for either very small code payloads (such as aws-lite), or scenarios where we are using the AWS SDK included in the Lambda image (e.g. @aws-sdk v3 raw in nodejs20.x). Coldstart latency increases as code payload sizes increase; this is most clearly observed with bundled SDKs.

Initialization latency

Initialization latency measures the impact of each SDK on the initialization phase of the Lambda lifecycle, including static analysis and execution of any code outside the scope of the Lambda handler.

Here we expect to see relatively similar values, as the performance benchmark has almost no static code or init-time execution.

Import / require

Here we measure the impact of importing / requiring each SDK. Ideally, all import / require operations should be sub-100ms to ensure fast responses in customer hot-paths.

It is important to note that import / require times are tied to individual services. In this benchmark, only the DynamoDB service client is imported. In real world use your business logic may rely on multiple AWS services – each of which necessitating additional imports, thereby compounding overall response latency.

Instantiate a client

Here we measure the impact of instantiating a new SDK client – a necessary step before making any service API calls. Ideally all operations should be sub-50ms to ensure fast responses in customer hot-paths.

DynamoDB - read one 100KB row

Here we measure the latency associated with reading a single 100KB row from DynamoDB, and parsing and returning results. All reads are identical across SDKs.

DynamoDB - write one 100KB row

Here we measure the latency associated with writing a single 100KB row into DynamoDB. All writes are identical across SDKs.

Peak memory consumption over Lambda baseline

Peak memory consumption measures each SDK’s peak memory usage throughout the above four steps (import / require, instantiation, read, and write).

To make it easier to assess the memory impact of each SDK, the graph is presented as a value over (thus, not including) the Lambda Node.js baseline. Baseline memory consumption would be expected to include Node.js itself, Lambda bootstrap processes, etc. The memory baseline used always corresponds to the equivalent peak memory of the control test (e.g. aws-lite peak memory p95 - control peak memory p95 = peak memory over baseline p95).

Time to respond, not including coldstart

Time to respond measures the total execution time of each SDK, not including coldstart or initialization. In real-world usage, Lambda coldstarts are usually less common than warm invocations, so this metric illustrates the most common case for most applications. Ideally all times should be sub-1000ms to ensure fast responses in customer hot-paths.

Total time to respond, including coldstart

Total time to respond measures the total execution time of each SDK, including coldstart or initialization. In real-world usage, this metric represents a normalized “worst case” response time. Ideally all times should be sub-1000ms to ensure fast responses in customer hot-paths.

Latest data

If you’d like to dig deeper, here’s the data from the latest performance run:

• Raw, unparsed results
• Parsed results