Real-time Notifications on WebUI

Egnyte handles petabytes of data originating from millions of users who interact with the platform using diverse devices. While a user is connected to Egnyte, it is very important to provide them with a superior user experience. 

Egnyte supports different types of clients such as:

• iPhone and Android;
• Desktop Applications on Mac and Windows; and
• Web User Interface (WebUI).

Our platform is used by teams of different sizes. Consider a scenario where you are connected to the Egnyte Collaboration Platform using a browser (WebUI). At the same time, your colleague is also browsing through the same folder and creates a new file or folder in it using a different client. Normally, you will only know about this activity if you refresh the web page and the latest data is pulled from the server.

Another option is to see notifications that are sent to the client in real-time, which are called real-time notifications.

This blog details how we deliver real-time notifications. At Egnyte, everything is about scale. We not only have to deliver these events in near-real time, but we also have to do this for thousands of clients and millions of updates daily.

To support this process we have built and deployed a component that we internally call RED—an acronym for Real-time Event Delivery. ‍

RED (Real-Time Event Delivery)

RED is a cloud service developed to serve real-time notifications to clients.

When a client is connected to RED, it holds a full duplex web socket connection through which the server can initiate a conversation as a Push Notification to the client.

• Connection with RED enables clients to receive real-time notifications for the events in subscribed folders.
• It works with the following sequence:
  • Filter Module: RED filters every event occurring within a tenant at Egnyte and checks if it is eligible for any connected client. If an “Event” is required to be delivered to any “Client”, it stores the event details in the RED Message Store and notifies the Transport Module. This filtering is essential to Egnyte’s granular permission model.
  • Transport Module: Filtered events are communicated to relevant clients over a web-socket connection.

To implement Real-Time Notifications support on WebUI, it requires the following:

1. WebUI should be able to connect to RED;
2. RED should be able to connect, authenticate, and manage WebUI connections.

 RED Overview

‍Components

Web User Interface (WebUI)

This feature is implemented for the WebUI and includes the following modules:

• Connection Manager
  • Every tab in the browser has a new connection with RED.
  • A periodic PING is sent to the server and a PONG is expected from the server in response.
  • Optimized Retry mechanism is implemented: 
    • Exponential Backoff: A way to retry connections without repeated immediate retries.
    • Random Starting Point: The exponential backoff is started after a random delay to avoid congestion.
• Event Listeners
  • Once the connection is established, it listens to incoming events. 
  • The Event Listener is responsible for handling several types of events (open, message, error, close) appropriately.
  • When any of the desired events are notified to the client, a notification popup is shown in the browser.‍

RED

• Web-socket Connection with WebUI.
• Authentication support for WebUI connections:
  • To achieve this, we have introduced short-lived access tokens that work only for WebUI.
  • These tokens are generated by Egnyte Auth Engine on Request by WebUI.
  • The token is then passed to RED along with an Authentication Message.
• A connection timeout mechanism is implemented:
  • If the connection is unauthorized (T1).
  • If there is no activity over the connection (T2).
• A PING / PONG mechanism is implemented and is consumed by Clients periodically to keep the connection alive.‍

Flow

Connection (Authentication and PING/ PONG)

• WebUI establishes a web socket (unauthenticated) connection with RED:
  • If no authentication message is received, the connection is terminated by the server.
• Over the secure connection, the client then sends the authentication message.
• On an authentication message, the server authenticates the client.
• On a PING message, the server responds with a PONG message to the client.
• If there is no message from the client in 2 minutes, the server disconnects the connection.

‍Event Processing

• When any event takes place in the client subscription context, it is written in a Redis pub-sub.
• RED is subscribed to this redis pub-sub, hence RED is notified about this event.
• RED Instance checks the applicability of this event with each of its connected clients.
• It then delivers the event to the relevant clients over the secure web socket connection.

Scale

The Egnyte platform deals with millions of users. Handling so many users and connections at a time is very important. For RED 2 auto-scaling provisions are made to work with a large number of connections efficiently and optimally.

• L1 haproxy replicas are auto-scaled based on new connections per replica.
• RED nodes are auto-scaled based on the CPU consumption of the existing nodes. 

Monitoring

At Egnyte, we are firm believers in observability. For RED, we capture many useful metrics that help us analyze and evaluate the service's performance better.

Some of the important metrics are: 

• Total Red connections.
• Events sent to clients.
• Number of service replicas.‍

Conclusion

With Real-time Notifications, we've taken significant strides to improve user interaction with the Egnyte collaboration platform. The user now receives instantaneous information within the same browsing session without any need for manual refresh or data pulling.

All in all, this innovative implementation of push notifications significantly removes the user's burden of continuously checking for updates. It encourages collaborative work by ensuring every member is promptly informed of any changes. It's a step forward in Egnyte's pursuit of making the user experience more interactive and intuitive at an unmatched scale.

‍