How WebAssembly Benefits from SolidQueue for Background Processing

1. The Need for Background Processing in WebAssembly-Driven Rails Apps

WebAssembly (WASM) is incredibly efficient at running CPU-intensive tasks in the browser.

However, when dealing with large-scale computations—such as batch processing, AI inference, or large dataset transformations—executing everything on the client side isn't always practical. This is where SolidQueue comes in.

SolidQueue allows us to offload heavy WebAssembly computations to the backend without blocking the main Rails thread.

Instead of making users wait for long-running operations, SolidQueue handles these tasks asynchronously and notifies the front end via Turbo Streams when the results are ready.

2. Setting Up SolidQueue in a Rails 8 App

To integrate SolidQueue into our Rails 8 app, we first need to install and configure it. Since we are using Rails 8, SolidQueue is already part of the framework, and no additional gems are required.

a) Run the following commands to install and set up SolidQueue:

> bundle add solid_queue
> bin/rails solid_queue:install

This will configure Solid Queue as the production Active Job backend, create the configuration files config/queue.yml and config/recurring.yml, and create the db/queue_schema.rb. It'll also create a bin/jobs executable wrapper that you can use to start Solid Queue.

b) Run the migrations:

rails db:migrate

Once you've done that, you will then have to add the configuration for the queue database in config/database.yml. If you're using SQLite, it'll look like this:

default: &default
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  <<: *default
  database: storage/development.sqlite3

test:
  <<: *default
  database: storage/test.sqlite3

production:
  primary:
    <<: *default
    database: storage/production.sqlite3
  cache:
    <<: *default
    database: storage/production_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/production_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/production_cable.sqlite3
    migrations_paths: db/cable_migrate

c) Configure and run SolidQueue in development environment:

Calling bin/rails solid_queue:install will automatically add config.solid_queue.connects_to = { database: { writing: :queue } } to config/environments/production.rb.

In order to use Solid Queue in other environments (such as development or staging), you'll need to add a similar configuration(s).

For example, if you're using SQLite in development, update config/database.yml as follows:

development:
+ primary:
    <<: *default
    database: storage/development.sqlite3
+  queue:
+    <<: *default
+    database: storage/development_queue.sqlite3
+    migrations_paths: db/queue_migrate

Next, add the following to config/environments/development.rb

  # Use Solid Queue in Development.
  config.active_job.queue_adapter = :solid_queue
  config.solid_queue.connects_to = { database: { writing: :queue } }

d) Once you've added this, run db:prepare to create the Solid Queue database and load the schema.

e) Finally, in order for jobs to be processed, you'll need to have Solid Queue running.

In Development, this can be done via the Puma plugin as well. In config/puma.rb update the following line:

#You can either set the env var, or check for development
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"] || Rails.env.development?

Note about Action Cable: If you use Action Cable (or anything dependent on Action Cable, such as Turbo Streams), you will also need to update it to use a database.

In config/cable.yml

development:
-  adapter: async
+ adapter: solid_cable
+  connects_to:
+    database:
+      writing: cable
+  polling_interval: 0.1.seconds
+  message_retention: 1.day

In config/database.yml

development:
  primary:
    <<: *default
    database: storage/development.sqlite3
+  cable:
+    <<: *default
+    database: storage/development_cable.sqlite3
+    migrations_paths: db/cable_migrate

To know more about how to setup or configure your SolidQueue then you can explore its documentation.

3. Offloading WebAssembly Computations to SolidQueue

WebAssembly is designed to run in the browser, making it ideal for performance-intensive tasks like image processing, cryptographic operations, or complex calculations.

However, when WebAssembly runs in the browser, we might need to prioritize, queue, and control how WebAssembly tasks execute to avoid blocking the UI.

This is where SolidQueue comes in—while typically used for backend job processing, we can use it for managing WebAssembly execution within the browser itself.

Why Use SolidQueue for WebAssembly in the Frontend?

Prevents UI Freezing – If WebAssembly runs heavy computations directly, the browser's UI can freeze. Instead, we queue computations using SolidQueue to ensure smooth performance.
Manages WebAssembly Tasks Efficiently – WebAssembly modules can be queued and executed in order, ensuring one task completes before another starts.
Improves Performance in SPAs & PWAs – A StimulusJS controller with SolidQueue ensures WebAssembly runs asynchronously, making it efficient for frontend apps.

4. Example

Let’s consider an example where a user uploads an image, and we need to apply WebAssembly-powered filters asynchronously.

However, we do NOT want our user to wait for the filter to be processed thus we offload the response of the filter WASM Module to a queue so that our HTML page can be updated asyncronously once the filter gets applied on the image.

a) Create a WebAssembly module, image_processor, for image processing

use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn apply_filter(image_data: &mut [u8]) -> Vec<u8> {
    for i in (0..image_data.len()).step_by(4) {
        let r = image_data[i] as f32 * 0.8;
        let g = image_data[i + 1] as f32 * 0.7;
        let b = image_data[i + 2] as f32 * 0.9;

        image_data[i] = r as u8;
        image_data[i + 1] = g as u8;
        image_data[i + 2] = b as u8;
    }
    image_data.to_vec()
}

b) Compile it:

wasm-pack build --release --target web

c) Move the compiled WebAssembly into Rails:

> mkdir -p app/assets/wasm
> mv lib/wasm/image_processor/pkg/wasm_processor_bg.wasm app/assets/wasm/
> mkdir -p app/javascript/wasm
> mv lib/wasm/image_processor/pkg/wasm_processor.js app/javascript/wasm

d) Load the WASM Module using a Stimulus Controller

Create app/javascript/controllers/image_processor_controller.js and copy the content in it from below:

import { Controller } from "@hotwired/stimulus";
import { postData } from "../utils/api"; // ✅ Helper for making POST requests
import init, { applyFilter } from "/wasm/image_processor.js"; // ✅ WebAssembly module

export default class extends Controller {
  static targets = ["image", "status"];

  async process() {
    await init(); // Initialize WebAssembly

    const imageElement = this.imageTarget;
    const canvas = document.createElement("canvas");
    const ctx = canvas.getContext("2d");

    canvas.width = imageElement.width;
    canvas.height = imageElement.height;
    ctx.drawImage(imageElement, 0, 0);

    const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
    const processedImageData = applyFilter(imageData.data); // ✅ Process in WebAssembly

    ctx.putImageData(new ImageData(processedImageData, canvas.width, canvas.height), 0, 0);
    const processedImageUrl = canvas.toDataURL("image/png"); // ✅ Convert to PNG

    this.sendImageToRails(processedImageUrl);
  }

  async sendImageToRails(imageData) {
    this.statusTarget.innerText = "Sending to Rails...";

    try {
      await postData("/images/process", { image: imageData }); // ✅ Send to Rails
      this.statusTarget.innerText = "Processing...";
    } catch (error) {
      console.error("Failed to send image:", error);
      this.statusTarget.innerText = "Error processing image.";
    }
  }
}

e) Add app/javascript/utils/api.js:

export async function postData(url, data = {}) {
  const response = await fetch(url, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-CSRF-Token": document.querySelector("meta[name='csrf-token']").content
    },
    body: JSON.stringify(data)
  });

  if (!response.ok) {
    throw new Error(`HTTP error! status: ${response.status}`);
  }

  return response.json();
}

f) Create Rails Controller app/controllers/images_controller.rb to Handle Processed Image:

class ImagesController < ApplicationController
  def process
    image_data = params[:image] # ✅ Base64-encoded processed image
    SolidQueue.enqueue("processed_images", { image: image_data }) # ✅ Enqueue image in SolidQueue
    head :accepted
  end
end

g) Create a SolidQueue Consumer to Send Image via Turbo Streams app/consumers/processed_image_consumer.rb:

class ProcessedImageConsumer < SolidQueue::Consumer
  queue_name "processed_images"

  def consume(data)
    processed_image_data = data["image"]

    # ✅ Save processed image (Rails does NOT process it)
    image = Image.create(file: Base64.decode64(processed_image_data.split(",")[1]))

    # ✅ Turbo Stream sends processed image back to UI
    Turbo::StreamsChannel.broadcast_replace_to(
      "image_list",
      target: "processed_image",
      partial: "images/processed_image",
      locals: { image: image }
    )
  end
end

h) Update the Rails View app/views/images/show.html.erb:

<div data-controller="image-processor">
  <img src="<%= @image.file_url %>" data-image-processor-target="image" id="processed_image" />
  <button data-action="click->image-processor#process">Apply Filter</button>
  <p id="status" data-image-processor-target="status"></p>
</div>

i) Modify app/views/images/_processed_image.html.erb:

<img src="<%= image.file_url %>" id="processed_image" />

j) User Workflow of the example:

1️⃣ User clicks "Apply Filter" → WebAssembly processes image in the browser.
2️⃣ Stimulus controller sends the processed image to Rails via an HTTP request.
3️⃣ Rails queues the image in SolidQueue (Rails does not process it).
4️⃣ SolidQueue consumer retrieves the image and broadcasts it to the UI via Turbo Streams.
5️⃣ Turbo Stream updates the UI instantly with the processed image.

Note:- The above example is NOT tested as yet. So, it might not work as expected. The User Workflow also might not be correct, however the example is good enough to explain where SolidQueue can be used with WebAssembly.