Have you seen this error when trying to deploy a TypeScript and NPM based Lambda to AWS?
An error occurred (InvalidParameterValueException) when calling the UpdateFunctionCode operation: Unzipped size must be smaller than 262144000 bytesThis comes from an important AWS Lambda limit, related to the “Deployment package size”
To illustrate what we’ll be doing to solve this, let’s start by creating a very simple Lambda, that gets an ID from the Lambda event parameters and read a DynamoDB record by this ID:
import DynamoDB = require('aws-sdk/clients/dynamodb');
import {GetItemInput} from 'aws-sdk/clients/dynamodb';
const ddb = new DynamoDB({apiVersion: '2012-08-10'});
export const handler = async (event: { id: string }) => {
const params: GetItemInput = {
TableName: process.env.TABLE_NAME as string,
Key: {
'id': {N: event.id}
},
};
// Read item from DynamoDB
const result = await ddb.getItem(params).promise();
// Return the item as Lambda result
return result.Item;
}Simple, right?
Let’s try compiling this .ts file to .js and deploying the Lambda folder along with the node_modules dependencies (which mostly contains aws-sdk and a few other sub-dependencies).
We have only written 19 lines of code, yet our deployment size (the .zip file) is already 15MB. The unzipped version is over 100MB. We quickly realize that if use one or two more dependencies while writing our Lambda code (e.g. Lodash or some useful Math library), we will quickly reach the AWS Lambda deployment size limit of 250MB unzipped source code (including node_modules).
But what is the solution?
Webpack
Using Webpack to shrink a TypeScript file and only its required third party dependencies, into a single, minified JavaScript file
First, let’s install Webpack and a few dependencies:
npm i webpack webpack-cli ts-loaderCreate a webpack.config.js file at the root of your project:
const path = require('path');
const fs = require('fs');
// Search for Lambdas in this folder
const dir = path.resolve(__dirname, '/lambdas/');
const handlers = fs.readdirSync(dir).filter(function (file) {
// Get only .ts files (ignore .d.ts)
return file.match(/(^.?|\.[^d]|[^.]d|[^.][^d])\.ts$/);
});
// Collect Webpack suitable entries
// This object will contain a key=>value array of
// filename => file absolute path
// which is what Webpack expects
// @see https://webpack.js.org/concepts/entry-points/
const entries = {};
handlers.forEach(handler => {
// Remove extension from filename
const filenameWithoutExt = handler.replace('.ts', '');
entries[filenameWithoutExt] = dir + handler;
});
module.exports = {
// https://webpack.js.org/concepts/entry-points/
entry: entries,
// https://webpack.js.org/configuration/mode/
mode: 'production',
// https://webpack.js.org/configuration/target/
target: 'node',
module: {
rules: [
{
test: /\.tsx?$/,
use: {
// Pass .ts files through this tool
loader: 'ts-loader',
options: {
// Ignore potential TypeScript compilation errors
transpileOnly: true,
}
},
exclude: /node_modules/,
},
],
},
resolve: {
modules: [
dir,
path.resolve(__dirname, 'node_modules'),
],
extensions: ['.tsx', '.ts', '.js'],
},
output: {
libraryTarget: 'umd',
// Where to write the resulting compiled .js files
path: path.resolve(__dirname, 'dist/lambdas'),
// Define a specific naming scheme if you need one
filename: "[name].js"
},
};
The configuration file mostly defines where will Webpack look for files to compile (./lambdas in this case), how will it compile them (using “ts-loader”) and where to store them (./dist/lambdas).
Now, when we run “npx webpack” and inspect the output folder ./dist/lambdas, we should see our main Lambda handler with the 19 lines of code above, compiled to JavaScript and prefixed with all of the actual libraries or methods that it uses from node_modules, making node_modules no longer a deployment requirement inside the .zip file to be uploaded to AWS Lambda.
Let’s try deploying only the “artifacts” from the ./dist/lambda folder now.
The unzipped size is not that much bigger (700kb). These is a significant difference between <1MB and the original deployment size of 100MB+.
Summary
Using the above approach, you should be able to safely keep developing new Lambdas and start using third-party node_modules based libraries without worrying about AWS Lambda deployment size limits.
Using Webpack compilation as a step between writing code and deploying it, will make sure your deployments grow predictably in size, relatively to the actual code you write and use (Webpack strips away unused code using a “tree-shaking” mechanism) – not relative to code you add using “npm install …”.
Happy Lambda coding.
Are you our your company having issues with integrating AWS services, AWS CDK, Terraform or Docker? Drop me a line.
