The rise of Kubernetes...

The rise of... containerized applications...

The rise of... RAM, Disk, CPU, and Billing Statements...

Living in this so called microservice storm taught me that everyone wants their stuff fancy. It's not the era of monolithic apps anymore, or so being said by those who oblivious that Github is still a monolithic app. But eh, into actually handling the apps in VM or even bare metal servers is never going to be a great experience. In the midst of the rise of containerization, to isolate our app (and ourselves) from the outside world, we now have the habit of adding Dockerfile after .gitignore when starting our repos.

With that said, I also jumped into the hype. Because why not? I feel a little bit uneasy looking on my DigitalOcean droplet running with only 5% CPU usage, basically doing nothing. And so I started looking at my apps that are running there.

I have : A Discord Bot running in Node.js, A Postgres Service running on top of systemctl, and what else... nothing i guess. As it's not something so complicated, I decided to just... containerize it. I kinda addicted on how Docker Compose handles the lifecycle also, so yea, let's do this.

Containerizing The Discord Bot

As I have probably cheated with using Node.js to develop my bot, it's not much of a great challenge to dockerize. The only hard stuff is the existence of Typescript, which could be a blessing and also a curse. I swear, the amount of compilation error between local and remote is just weird, even with the same major Node version and the same tsconfig... configuration. Did containerizing solve my problem? Partially. Mostly. It decreased the chance of failed build because of weird compilation error ever since I decided to just containerize the app. I don't want to jinx it, but I know it will come, sooner or later.

My bot actually has 2 parts : Bot client and Web server. But not a big problem as they both lie on the same parent directory, and also written in Typescript, so it shouldn't give any obstacle while compiling. But I use Prisma Io, which is the best ORM so far btw, to handle the database stuffs. As it's kinda detached with the actual source code, an additional step which to build the Prisma client is done after the compilation. Additionally, to reduce the size, multi-stage build is used, although it doesn't really do much.

FROM node:16-buster-slim as builder
WORKDIR /tmp
COPY package.json tsconfig.json /tmp/
RUN npm install
COPY ./src ./src
RUN npm run build


FROM node:16-buster-slim
WORKDIR /usr/src/app
COPY package.json ./
RUN apt-get update && apt-get install openssl -y
RUN npm install
COPY --from=builder /tmp/build ./build
COPY ./prisma ./prisma
RUN npx prisma generate
EXPOSE 8000
CMD ["node", "build/index.js"]

As you can see, the builder stage just build whatever lies in the source folder, then copies the result to the end image later on. Prisma client is being built over there also, and because it somehow asked for weird SSL thingy, additional apt package is added to complement.

Now this should be enough for the bot itself. But how about the database?

Containerizing The Database : Postgres

At first I used Postgres for the database. Because I kinda want to do it faster, I just use the docker image and run one locally. Then I forgot that the storage is only persist as long the container exist, so better to attach volumes eh. My requirements aren't that hard (I don't want to be too hard to myself~), so it just went quickly. I used Docker Compose to better manage the containers, as to said.

version: "3.5"


services:
    db:
        container_name: [REDACTED]
        image: postgres
        restart: always
        environment:
            POSTGRES_USER: [REDACTED]
            POSTGRES_PASSWORD: [REDACTED]
            POSTGRES_DB: [REDACTED]
        ports:
            - "5432:5432"
        volumes:
            - type: volume
              source: pg_data
              target: /var/lib/postgresql/data
        networks:
          - [REDACTED]

Although each compose deployment will spawn it's own network, I just go with using the existing network. Mostly because the default network name is too verbose, so I have something like a single-word name for the network.

Now this won't give any much features, like there won't be higher security level, no SSL, just leaving everything default to Postgres. But it was enough.

...up until I wanted to add features that involves DDL operation.

As this is only a hobby project, there is simply too much constraint from relational database. Altering the table would need additional operation for the existing values, null handling, changing the data type, and just too much of things I worry that I ended up not developing anything. So I decided to move to NoSQL instead. I picked mongoDb because yes.

Containerizing The Database : MongoDb

Going with mongodb, it's quite similar with how I did it with postgres. Just create another service in the docker compose file, add environment variables, add volumes, and others, and done!

It's simple!

I wished.

Deploying with the default values will make the the database counts as "Standalone". Mostly, this is already enough. You can do the basic CRUD operation already with this setting.

But Prisma wanted to use transaction API, and it's not supported if it's not a "Replica Set" deployment, so I had to work on that.

After an hour diving into stackoverflow and the mongodb community forums, I found two things :

It's possible to make the deployment "think" as replica sets by doing healthcheck .
It's not possible to not use keyfile, even if you don't need SSL, as it keeps complaining that keyfile is misconfigured.

So, with this information, I built my service for mongodb like this.

    mongo:
      container_name: [REDACTED]  
      image: mongo
      restart: always
      hostname: mongodb
      environment:
            MONGO_INITDB_ROOT_USERNAME: [REDACTED]
            MONGO_INITDB_ROOT_PASSWORD: [REDACTED]
            MONGO_REPLICA_SET_NAME: rs0
      ports:
            - "27017:27017"
      volumes:
            - type: volume
              source: mongo_data
              target: /data/db
            - ./:/opt/keyfile/
      healthcheck:
        test: |
          test $$(mongosh --quiet -u [REDACTED]-p [REDACTED] --eval "try { rs.initiate({ _id: 'rs0', members: [{ _id: 0, host: 'mongodb' }] }).ok } catch (_) { rs.status().ok }") -eq 1
        interval: 10s
        start_period: 30s
      command: "--bind_ip_all --keyFile /opt/keyfile/keyfile --replSet rs0"
      networks:
        - kureya

And the keyfile is just simple file consisting of output from the following commands

openssl rand -base64 756 > keyfile
chmod 999 keyfile
chgrp 999 keyfile

Such an open permission, but no problem as it's going to live inside the container only.

Something to point out as this is a single-node replica set, in the healthcheck part there is rs.initiate() function call, which tells the initial database instances that is going to be a part of the replica set. For the _id , it has the same name with the replica set name environment variable, and the only member in members array has the host pointed to the hostname configuration for the service. This is to speak that it points to itself and therefore the "replica set" is only a myth. Such fate, well suited for my poor 1GB RAM VM.

With that, it's done. I redeployed my bot environment into docker compose, and the next time I want to update stuff, it's all just to edit the YAML file. Might need to properly extend this to handle secrets and stuff, but I think it's already good enough. The definition for the bot is probably the simplest one, so no need to put it here (just specifying image and environment variable, oh also the network).

Loading...

There will be awesome contents here

Into Actually Containerizing My Bot Ecosystem