Today, we delve into the deployment of a cloud-based web application for text extraction from images, focusing on the streamlined deployment process made possible through Ansible. This project, developed as part of a university seminar, demonstrates how complex infrastructure and application setups can be automated and managed with a single command.
Table of Contents
Introduction
Deploying a scalable and efficient cloud-based application often involves numerous steps and configurations. However, with the power of Ansible, we have streamlined this process into a single, executable command. This post will walk you through the architecture, challenges, and solutions that made this possible.
System Architecture and Design
The architecture of our application is designed to leverage various cloud service models, ensuring scalability, reliability, and performance. Key components include:
- Frontend and Backend Integration: Developed using Nuxt.js, enabling seamless integration within a monorepo architecture.
- Load Balancer and Host VMs: Ensures high availability and distributes traffic across multiple instances.
- Caching and AI Processing: Uses Redis for caching and Google Vertex AI for efficient text extraction.
This diagram illustrates the overall system architecture, showing the interaction between various components such as the load balancer, host VMs, Redis instance, and Vertex AI within the Google Cloud Platform.
Key Challenges and Solutions
Infrastructure Provisioning with Terraform
Challenge: Managing complex cloud infrastructure in a consistent and scalable manner.
Solution:
- Modular Terraform Configuration: Structured Terraform files for clarity and maintainability, enabling easy scaling and updates.
- Integration with Ansible: Terraform scripts are invoked within the Ansible playbook, automating the provisioning of necessary resources like VMs, networking settings, and data storage.
Passing the User for SSH Keys: Including the user in the Terraform configuration was crucial for adding SSH keys to the VMs, allowing Ansible to access these instances.
- name: Deploy resources to google cloud
hosts: localhost
connection: local
gather_facts: false
tasks:
- name: Get current username
local_action: command whoami
register: username
- name: Apply the terraform configuration
community.general.terraform:
project_path: "{{ playbook_dir }}/../terraform"
state: present
variables:
project_id: "{{ service_account.project_id }}"
user: "{{ username.stdout }}"
register: deployed_tf
This task applies the Terraform configuration, provisioning the required cloud infrastructure. The user variable is passed to ensure SSH keys are added to the VMs.
resource "google_compute_instance" "host_vm" {
count = 2
name = "host-vm-${count.index}"
machine_type = "e2-micro"
zone = var.zone
// ...
metadata = {
ssh-keys = "${var.user}:${file("/home/${var.user}/.ssh/id_rsa.pub")}"
}
// ...
}
Deployment and Operational Management with Ansible
Challenge: Automating the deployment process to ensure all services and dependencies are correctly configured and deployed.
Solution:
- Dynamic Inventory Management: Ansible dynamically manages the inventory, adding provisioned VMs as targets for subsequent tasks.
- Single Command Execution: The entire deployment process, from infrastructure setup to application deployment, is executed with a single Ansible command, simplifying operations.
Initially, I wanted to construct the inventory using the following Terraform resource:
resource "local_file" "AnsibleInventory" {
content = <<-EOF
[host-vm]
${google_compute_instance.host_vm.network_interface.0.access_config.0.nat_ip}
EOF
filename = "${path.module}/../ansible/inventory.ini"
}
And then loading the inventory via ansible.builtin.meta: refresh_inventory.
However, this approach caused all variables from before to be lost because loading a new inventory resets all variables, even in the localhost scope.
- name: Save instance IPs
ansible.builtin.set_fact:
redis_instance_ip: "{{ deployed_tf.outputs.redis_instance_ip.value }}"
host_vm_ips: "{{ deployed_tf.outputs.host_vm_ips.value }}"
load_balancer_ip: "{{ deployed_tf.outputs.load_balancer_ip.value }}"
This task saves the IP addresses of the Redis instance, host VMs, and load balancer from the Terraform outputs.
- name: Add host_vm_ips to inventory
ansible.builtin.add_host:
name: "{{ item }}"
ansible_ssh_common_args: '-o StrictHostKeyChecking=no'
groups: host-vm
with_items: "{{ host_vm_ips }}"
This task dynamically adds the host VMs to the Ansible inventory, enabling subsequent tasks to target these instances for configuration and deployment. The main advantage is that this does not reload the entire inventory, thus keeping variables saved in other hosts.
Continuous Integration and Deployment (CI/CD)
Implementing a robust CI/CD pipeline is crucial for ensuring smooth operations, automated testing, and seamless deployment of updates. This section details the configuration of GitHub Actions for building, testing, and deploying our cloud-based text extraction application.
CI/CD Pipeline Overview
Our CI/CD pipeline is divided into two main workflows:
- Build and Test Service: This workflow runs tests on the service to ensure code quality and functionality.
- Build and Upload Docker Image: This workflow builds the Docker image and uploads it to the GitHub Container Registry.
Build and Test Service
The test.yml workflow is responsible for building and testing the service. It runs on every push to the master branch and can also be triggered manually.
# ...
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v3
with:
version: 8
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
- run: pnpm install
working-directory: ./service
- run: pnpm test
working-directory: ./service
This executes the tests in the ./service directory.
Build and Upload Docker Image
The docker.yaml workflow is responsible for building the Docker image of the service and pushing it to the GitHub Container Registry. This workflow runs on every push to the master branch and can also be triggered manually.
# ...
steps:
- name: Set up Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Github Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: "{{defaultContext}}:service"
push: true
tags: ghcr.io/${{ github.repository_owner }}/image_text_extraction:latest
- Set up Buildx: The
docker/setup-buildx-action@v3step sets up Docker Buildx, a tool for building multi-platform Docker images. - Login to GitHub Container Registry: The
docker/login-action@v3step logs into the GitHub Container Registry using the repository owner's credentials and a GitHub token. - Build and Push Docker Image: The
docker/build-push-action@v5step builds the Docker image from the specified context (./service) and pushes it to the GitHub Container Registry with the taglatest.
Caching and Performance Optimization
Efficient caching and performance optimization are critical to ensuring that our cloud-based text extraction application remains responsive and scalable. In this chapter, we delve into how Redis caching is implemented to enhance performance and reduce latency, especially for repeat requests.
Hashing the Image Data
To uniquely identify each image and its associated text, we use a hashing function. This ensures that each image is represented by a unique key in the cache.
import { hash } from "ohash"
// ...
const key = hash(data)
For this we use the murmurhash3-based hash function of the wonderful unjs ecosystem.
Checking Redis for the Cached Text
Before processing the image for text extraction, we check if the text is already cached in Redis. If it is, we can return the cached result immediately, saving computational resources and time.
For this we first need to establish a connection to redis. To achieve this we use Nuxt's unstorage based connectors.
import redisDriver from "unstorage/drivers/redis"
export default defineNitroPlugin(() => {
const storage = useStorage()
// Mount the (env var driven) Redis driver only in production
if (process.env.NODE_ENV === "production") {
const driver = redisDriver({
base: "redis",
host: useRuntimeConfig().redis_host || "localhost",
port: useRuntimeConfig().redis_port ? parseInt(useRuntimeConfig().redis_port) : 6379,
password: useRuntimeConfig().redis_password,
})
storage.mount("redis", driver)
}
})
This is only used in production, since in development, we can rely on the nitro.devStorage option of nuxt.config.ts.
After this we only need to check the existence of the hash in redis:
const cachedText = await redis.getItem(key)
if (cachedText) {
// Create an instantly closing stream with the cached text
const stream = new ReadableStream({
start(controller) {
controller.enqueue(cachedText)
controller.close()
}
})
return sendStream(event, stream)
}
Here you can also see the third optimization technique which is streaming.
Using Streaming to increase perceived Speed
To enhance the user experience, we use streaming to process and send the text as soon as it becomes available. This makes the application appear faster as users begin receiving data almost immediately.
const stream = new ReadableStream({
async start(controller) {
for await (const chunk of res.stream) {
const text = chunk.candidates.reduce((acc, candidate) => {
return acc + candidate.content.parts.reduce((acc, part) => {
return acc + part.text
}, "")
}, "")
const isFinish = chunk.candidates.some(candidate => candidate.finishReason || candidate.finishMessage)
controller.enqueue(text)
if (isFinish) {
controller.close()
}
}
},
})
- Readable Stream: A ReadableStream is created to process the response from Vertex AI as it becomes available.
- Immediate Feedback: The controller.enqueue(text) method sends chunks of text to the client as they are received, providing immediate feedback and improving perceived speed.
Storing the Extracted Text in Redis
After the text is extracted from the image, it is stored in Redis for future use. This ensures that subsequent requests for the same image are served quickly from the cache.
res.response.then((response) => {
const text = response.candidates.reduce((acc, candidate) => {
return acc + candidate.content.parts.reduce((acc, part) => {
return acc + part.text
}, "")
}, "")
redis.setItem(key, text)
})
Lastly, the extracted text is stored in Redis using the unique hash as the key. This ensures that future requests for the same image can be served from the cache, improving response times.
Deploying to Systems
As a last step, the previously created Docker container is pulled and executed as a service on the host VMs.
- name: Pull docker image
ansible.builtin.command:
cmd: docker pull {{ container_image }}
register: pull_image
- name: Save redis_instance_ip from localhost
ansible.builtin.set_fact:
redis_instance_ip: "{{ hostvars['localhost']['redis_instance_ip'] }}"
when: redis_instance_ip is not defined
- name: Save service_account from localhost
ansible.builtin.set_fact:
service_account: "{{ hostvars['localhost']['service_account'] }}"
when: service_account is not defined
- name: Create Service file
ansible.builtin.template:
src: docker-service.j2
dest: /etc/systemd/system/{{ container_name }}.service
mode: "0644"
notify: Restart Service
This also "transfers" the variables from the localhost scope to the current machine's scope.
The service is created using a templated systemd service file for the Docker container, ensuring it runs as a managed service on the host VM.
# templates/docker-service.j2
[Unit]
Description=Image Text Extractor Docker Container
Requires=docker.service
After=docker.service
[Service]
Restart=always
ExecStart=/usr/bin/docker run -p 80:3000 -e NUXT_PROJECT_ID='{{ service_account.project_id }}' -e NUXT_PRIVATE_KEY_ID='{{ service_account.private_key_id }}' -e NUXT_PRIVATE_KEY='{{ service_account.private_key | replace("\n", "\\n") }}' -e NUXT_CLIENT_EMAIL='{{ service_account.client_email }}' -e NUXT_CLIENT_ID='{{ service_account.client_id }}' -e NUXT_UNIVERSE_DOMAIN='{{ service_account.universe_domain }}' -e NUXT_REDIS_HOST='{{ redis_instance_ip }}' -e NUXT_REDIS_PORT='{{ 6379 }}' --name {{ container_name }} {{ container_image }}
ExecStop=/usr/bin/docker stop {{ container_name }}
ExecStopPost=/usr/bin/docker rm -f {{ container_name }}
TimeoutStartSec=0
[Install]
WantedBy=multi-user.target
This is the systemd service file template for the Docker container, defining how the container should be managed by systemd. It also passes all relevant env variables to the container.
Conclusion
Deploying a cloud-based text extraction application has been made significantly simpler through the use of Ansible and Terraform. By condensing the entire process into a single ansible-playbook playbook.yml command, we have reduced the complexity and potential for error, ensuring a smooth and efficient deployment. This can also be improved further by integrating this into a CI to fully automate the process.
The source code for this project is available on GitHub