Listening to Firestore with Web Workers

Google's Cloud Firebase provides complete, well-documented SDKs for different web & mobile platforms as well as many server client libraries. That helps developers to use Firebase products such as Firestore in many ways.

Google's Cloud Firestore is a powerful NoSQL document database that enables developers to seamlessly integrate with Web and Mobile solutions. One of its key features is the capability of automatic data synchronization between the apps which provides applications to run in real-time. So, its usage is very popular in real-time applications such as chats, gaming, etc.

How It Works in Real-Time

The implementation behind Firestore's real-time data synchronization is SDK provided onSnapshot method. That is mostly similar to the long-polling mechanism which continuously listens to a document or a collection in the Firestore database.

The Problem in Web Apps

Here I discuss an issue that could lead to experiencing many sporadic issues when Cloud Firestore is integrated with a web application along with its real-time capabilities.

When a web application starts to listen to a Firestore resource, it adds a considerable CPU-intensive load to the single threaded javascript engine. Depending on the complexity, that could severely affect UI rendering and event listening functionalities that the main execution thread needs to prioritize.

Solution with Web Workers

Javascript's Web Workers API has greatly relieved web applications by undertaking more CPU-intensive tasks out of the main execution thread to their own thread running in the background.

So, offloading Cloud Firestore connection and real-time listeners to a web worker would be a good approach in achieving a smooth and reliable user experience for a web application.

Let's Walkthrough the Code

Let's move forward and look into the implementation of web worker. Here I'm going to use the Angular framework, Firebase JS 9 SDK, and Angular wrapper of plain web worker in demo code snippets.

Although I'm not going to explain in detail how web workers and the main thread of the web app communicate. The simple explanation is that the communication between the web app's main thread and workers is message-based. Where both sides listen to message events and both sides post messages to the other whenever needed.

Creating Web Worker

Using Angular CLI, we can easily generate the Web Worker stub with all other configuration files that need to be configured in an Angular project.

ng generate web-worker firestore

In this sample scenario, My Cloud Firestore DB will have a collection of documents named users, and each of the documents in the collection has a sub-collection named notifications which will listen to in real-time for any new notification documents that would be added or removed.

Here in the web worker stub, let's add two functions to initialize the Firebase connection with given configurations and the other one to subscribe for a collection (or a document) to listen for the real-time updates.

So, any of the messages received to the web worker will have a property named type where it will select the function to be invoked depending on the type of the message.

/// <reference lib="webworker" />

import {collection, doc, getFirestore, onSnapshot, query, where} from 'firebase/firestore';
import {initializeApp} from 'firebase/app';

let firestoreDb;

function init(firebaseConfig) {
    const app = initializeApp(firebaseConfig);
    firestoreDb = getFirestore(app);
}

function subscribeForUserNotifications(userId: string) {
  // Create the query to listen 
  const userRef = doc(collection(firestoreDb, 'users'), userId);
  const firestoreQuery = query(collection(userRef, 'notifications'));

  // Listen to the notification collection 
  let unsubscribe = onSnapshot(firestoreQuery, (querySnapshot) => {
        querySnapshot.forEach((document) => {
          let doc = document.data();
          // Send data back to main application as a message
          postMessage(doc);
    });
  });
}

addEventListener('message', ({ data }) => {
    switch (data.type) {
        case 'INIT':
            init(data.config);
            break;
        case 'SUBSCRIBE_USER':
            subscribeForUserNotifications(data.userId);
            break;
    }
});

Creating Wrapper Service

Now the initial functionality of the web worker to connect and listen for Firestore DB is done. But the question remains how we can invoke the worker and get real-time updated data into the components of our Angular application. For that, let's create a wrapper service that we use anywhere in the Angular app.

import {Injectable} from '@angular/core';

@Injectable({
    providedIn: 'root'
})
export class FirestoreNotificationWrapperService {

    private worker;

    constructor( ) { }

    async init(settings): Promise<void> {
       if (typeof Worker !== 'undefined') {
            // Add the correct path to the worker in your project
            this.worker = new Worker('../workers/firestore.worker', { type: 'module' });
            this.worker.onmessage = (message) => {
                // Implementation goes here when update received
                console.log('Message received from worker : ' + message.data);
            };

            this.worker.postMessage({type: 'INIT', config: settings});
        } else {
            // Good to have a fall back mechanism if browser not supported
            console.log('Web worker not supported in this browser');
        }
    }

    public async subscribeForUserNotifications(userId: string): Promise<void> {
        this.worker.postMessage({type: 'SUBSCRIBE_USER', userId: userId });
    }
}

Once the wrapper service is in place, we can initialize the worker and start to listen to the collection in Firestore.

let firebaseSettings = {
    apiKey: 'xxxxx',
    authDomain: 'xxxxx.firebaseapp.com',
    projectId: 'xxxx',
    storageBucket: 'xxxxx',
    messagingSenderId: 'xxxxx',
    appId: 'x:xxxx:web:xxxxx'
};

let userId = 'xxxxxxxxxxxx';

this.firestoreNotificationService.init(firebaseSettings).then(() => {
     this.firestoreNotificationService.subscribeForUserNotifications(userId);
});

Make It Observable

In real application requirements, the chances are high where multiple components of the application are interested in the updates that are received from Firestore DB and those components might need to process them differently. For that, the above wrapper service is not suitable enough since we cannot externalize the processing logic of the messages received. Therefore let's modify the above wrapper service by adding an observable Subject.

import {Subject} from 'rxjs';

private notificationSubject =  new Subject();

getNotifications$() {
      return this.notificationSubject.asObservable();
}

Here we create a new Subject and modify the onmessage function to publish messages to the subject created. Then any component can subscribe to the Firestore updates through getNotifications$() function and process them separately.

this.worker.onmessage = (message) => {
       this.notificationSubject.next(message.data);
};

Summary

Google's Cloud Firestore with its real-time synchronization capabilities makes it a good tool to solve many different business problems. But the end-to-end implementation should have a fine-grained design. Especially in single-threaded environments like web applications.

Therefore integrations to the Cloud Firestore database from web applications using a web worker seem to be a promising and reliable solution. But do web workers facilitate utilizing all the features that are supported by Firebase SDK? The answer to the question is yet to be explored.

Agile Snippets