AWS is... pricey.

AWS is... confusing.

It really does need you to fully dedicate your entire day to actually learn one service or two just to get the slight grasp of it. That's not including the amount of googling required to "put things up". Tutorials are sometimes outdated, sometimes irrelevant, and sometimes just too obfuscated? Idk, this is probably just my bad luck when reading them articles. It's even ironic because I already have certification up to "Advanced Developing on AWS".

Yeah, it really boils down to how you keep honing your skills rather than just getting that certificate and completely forgetting about it.

AWS is "Fancy"

So, you're like me. You want to use AWS because it seems "fancy" and can add into your portfolio. Although you probably know another way that's a lot easier that doesn't really require AWS, you just want to try this thing. Or you maybe have some kind of assignments that targets AWS.

Yeah, with how AWS is really "scoped" to "premium users", it's kinda hard to find some decent tutorials that covers the actual basic or straight-to-the-point approach. Okay, there might be good sources out there but I probably have missed it somehow.

So, let's say you want to just try things up. Let's start with one of the most simplest thing that probably hooked lots of web developers already : S3. Probably because you want to use it to host static files, or images, or even static website. There already lots of tutorial out there for that since it's pretty "simple". You just need to kind of upload stuff and set is as public and there you are.

But let's say you want to do more "fancy" and "complicated" thing. Let's say you want to add auto-resizer functionality to every image that you upload to S3. Here you are referenced to another service called Lambda. It's basically acts like a "lambda function" in functional programming, but in more fancy way. It can listen to events from other services, such as upload event to a specific S3 bucket. Moreover, you can actually do stuff like resizing that recently uploaded image(s). You're even more hooked to AWS and decided to learn things up.

You thought it was going to be easy.

"Just another Node.js programming, what can possibly go wrong?"

Well, AWS Documentation wants to know your location.

S3, Buckets, and Policies

S3 : Simple Storage Service

Spoiler alert : It ain't simple. Okay, so you can use this service to upload your files. Think of a virtual hard drive, but not really. I want to point out this first because it's where you gonna host your files and images. There are lots of things happening in S3 and a lot of other services in AWS that can confuse you a lot, as I did. So, I'm just going to make this reaaaalllllyyy straightforward.

Buckets

You're going to organize your stuff inside a "Bucket". Think of it as the partition on your hard drive. You need to create a bucket in order to put stuff. Just click that orange button with "Create Bucket" to proceed.

It's pretty self-explanatory. Put the bucket name, set the location where it will be hosted, set permissions and other. As you want to make this bucket publicly accessible, you might gonna uncheck all of these.

You thought that is what it needs to make your files publicly accessible? HA! Ain't so simple, heh?

Introducing, policies.

Policies

It's something you use to control how things going on between your files and everything around it. That includes the external access, public access, etc.

There are much to cover about policies as it really just a subset of another service called IAM, but since I don't want to bullshit too much, I'll just put this policy so you can just use it.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::[YOUR BUCKET NAME HERE]/*"
        }
    ]
}

Put this "Bucket Policy" in "Permissions" tab of your bucket.

Yay, now your stuffs should be accessible publicly. Open the info for your file and there should be Object URL and you can just share it and use it in your web.

For convenience, I'm telling you to create another bucket with the same configuration as above, but with different bucket name. We're going to use this bucket to put the output of the resized image. There is a reason why we're doing this.

Okay, Now what?

Lambda

Okay so, AWS Lambda. Sounds cool. But how to actually use it?

Open up your IAM console. Yes, IAM. We're going to create a Role here.

IAM Roles

You need to create a "Role" which is a convenient way to attach "Policies". You can use this role to give access and control how other things behaves across AWS.

Just go to "Role" tab on the sidebar, click on "Create Role", and things gonna get weird from here.

Pick "AWS Service" for the type of trusted entity
Choose "Lambda" for a use case.
Click on "Next: Permissions" on the bottom right.
You gonna need to search and put checkbox on these 3 policies.
AWSLambda_FullAccess
AmazonS3FullAccess
AWSLambdaBasicExecutionRole
Give role name. Since we're going to access S3 from Lambda, just give convenient name like "AllowLambdaAccessToS3", or something.

Now you have a new role, yay. We're going to attach this to our Lambda function later.

Lambda Function

It's "FaaS" or Function as a Service from AWS. You create a function that can be invoked when an event specified raised up, and well, you can do stuffs.

Go to your AWS Lambda console and create a function. Pick "Author from scratch" since it's more fun ~~and suffering~~. Give name and since I'm gonna cover this one in Node.js, pick Node.js for the runtime. I'm using 14.x at the moment of writing this.

On the "Permissions" section, use "Use an Existing Role" and use the role you created earlier.

Something like this should be sufficient. Then click "Create Function".

Now, we need to make this function listen to events from other services. As we want it to listen from S3 uploads, open the function and on the Function Overview, click on "Add trigger". Select S3 and pick your bucket you've created earlier. Check the bottommost checkbox about recursive invocation. It's just something about Lambda gonna execute again if you put files inside the same bucket Lambda is listening to. This is why we created another bucket as the destination, just to prevent from the recursive and infinite loop.

After this, you'll have Lambda listen to events from S3. Now it's the exciting part.

Well, exciting is an overstatement, so... painful part.

But I'm just gonna give you the code so it's less painful, tehe.

Open up a VSCode or whatever you like to use. Init a node project. Create index.js file.

We're going to use library called "sharp" which is already famous library for image processing in Node.js. But to safeguard it, you need to install the library in your node project with Linux specified as the target platform. Well, this is just to adjust with how we've configured Lambda (you should also saw "x86_64" and "ARM" when creating the function, but it's something for another day

npm install --arch=x64 --platform=linux sharp
npm install --arch=x64 --platform=linux aws-sdk -D

The code for index.js

const AWS = require("aws-sdk");
const path = require("path");
const S3 = new AWS.S3({
    signatureVersion: "v4",
});


const Sharp = require("sharp");


exports.handler = function (event, context, callback) {
    const SOURCE_BUCKET = "[PUT YOUR SOURCE BUCKET HERE]";
    const TARGET_BUCKET = "[PUT YOUR TARGET BUCKET HERE]";
    const RESIZE_WIDTH = 200;
    const objectKey = event.Records[0].s3.object.key;
    const outObjectKey = path.parse(objectKey).name;
    if (event.Records[0].eventName === "ObjectCreated:Put") {
        S3.getObject({
            Bucket: SOURCE_BUCKET,
            Key: objectKey,
        })
            .promise()
            .then((data) => {
                console.log("Resizing using Sharp");
                return Sharp(data.Body).resize(RESIZE_WIDTH).toFormat("png").toBuffer();
            })
            .then((buffer) => {
                console.log("Putting output to another bucket");
                return S3.putObject({
                    Body: buffer,
                    Bucket: TARGET_BUCKET,
                    ContentType: "image/png",
                    Key: `${RESIZE_WIDTH}_${outObjectKey}.png`,
                }).promise();
            })
            .then(() =>
                callback(null, {
                    status: 200,
                })
            )
            .catch((err) => callback(err));
    } else {
        callback("Not a ObjectCreated:Put event. Exiting...");
    }
};

You can explore the Sharp library to fit your needs. Now, up to actually use this code in Lambda. The code above will resize the image, append the width as the prefix for the filename, and mark is as PNG with the extension.

As you might know, Node project will have package.json, node_modules folder, and index.js file as entry point. Excluding yarn.lock or package-lock.json, create a zip file consisting of all the stuffs we've created (package.json, node_modules, index.js). And upload this zip file into the Code source section of Code tab.

If things going well, it should return success. Upload images in your source bucket and wait for a while before the resized image goes into the target bucket.

This is poggers. No need to write somehow complicated image upload/resizer using Express and Multer and other stuffs.

Yet, it comes with the cost of money. AWS ain't cheap, but for the free tier, it's nice to safely say Lambda free tier already enough for personal usage. Just remember that S3 gonna always cost you.

Loading...

There will be awesome contents here

AWS Shenanigans #1 : Image Resizing for S3 Uploads with Node.js