Element Call: Redefining conferencing for privacy, scale, and sovereignty

Summary

The video presents a detailed overview and technical deep dive into Element Call, a decentralized, federated conferencing solution built on the Matrix open communication protocol, focusing on privacy, scalability, and user sovereignty.

---

Key Technological Concepts and Features

1. Matrix Protocol Overview

• Matrix is an open, decentralized, federated, replicated, and persistent network for secure real-time messaging.
• It supports end-to-end encryption and verified identities, crucial for privacy and trust.
• Originally designed for messaging, Matrix requires additional primitives to support real-time communication (RTC) like voice and video calls.

2. Challenges with RTC on Matrix

• Matrix’s eventual consistency and distributed nature cause issues like race conditions and split-brain states that affect call reliability.
• Early Matrix calling supported one-on-one calls and peer-to-peer group calls but did not scale well for large groups.
• Existing backend technologies like Jitsi (MCU) are not fully compatible with Matrix’s decentralized model.
• To scale, Element Call adopted SFU (Selective Forwarding Unit) architecture, enabling large group calls while maintaining decentralization.

3. Matrix RTC Layer and New Primitives

• Introduction of Matrix RTC, a structured framework layered on Matrix for real-time communication beyond just calls (e.g., whiteboards, games).
• New Matrix primitives developed:
  • Sticky Events: Lightweight, encrypted events that reduce metadata and state resolution overhead, improving efficiency and reliability.
  • Slots: Logical containers managing RTC applications within a Matrix room, requiring moderator-level power to open/close.
  • RTC Membership: Represents participants/devices in a call and their supported RTC transports.
• Support for pluggable RTC transports, currently using LifeKit (a WebRTC wrapper), with plans for future transports like WebTransport.

4. Matrix RTC Encryption Model

• End-to-end encryption supports subgroup addressing within rooms for calls.
• Uses individual sender keys rotated on membership changes for forward secrecy and post-compromise security.
• For very large calls, a shared key approach is optionally used.

5. Element Call Product Features

• Supports various call scenarios:
  • Telephone-style calls with ringing.
  • Ad hoc “huddles” inside existing rooms for spontaneous group discussions.
  • Persistent social rooms akin to Discord hangouts.
• Guest access and link sharing for external participants without Matrix accounts.
• Scales to millions of meeting minutes per month, with stable performance and low bug reports.
• Multi-SFU architecture distributes load fairly among home servers, improving resource use and privacy.
• Intelligent SFUs support simulcast streams with adaptive quality to optimize bandwidth comparable to MCU systems.
• Can host up to 500 participants on a single AWS instance with horizontal scaling via Kubernetes.

6. User Experience and Design (Presented by Aaron, Senior Designer)

• Separate UX flows for one-to-one calls and group calls on mobile and web.
• Features include:
  • Video/voice toggle with smooth switching.
  • Minimized call banner (instead of picture-in-picture) to encourage focus.
  • Call controls: mute, camera switch, raise hand, emoji reactions.
  • Persistent presenter tile in group calls.
  • Lobby for pre-join muting or camera off.
  • Participant list toggle and full-screen modes.
  • Call rating and feedback after calls.
• Desktop Electron app supports embedded widgets and plans for pop-out windows and picture-in-picture (PIP).

7. Ecosystem and Integration

• Matrix RTC applications are designed to be implemented as widgets for easy client integration.
• Reference implementation provided via the Element Call widget.
• Demo shown integrating Element Call widget into the Cine client with minimal code changes.
• SDK support available in Synapse (server), JS SDK, with Rust and others forthcoming.

8. Future Outlook

• Matrix RTC is considered production-ready with ongoing improvements.
• Further UI enhancements and feature rollouts planned.
• Additional talks and deep dives scheduled for the conference.

---

Summary of Q&A Highlights

• Desktop app supports embedded widgets; pop-out windows are planned but not immediate priority.
• Restricted users in multi-SFU setup can only subscribe to media, not publish.
• Sharing system audio during screen sharing is difficult due to echo cancellation and OS privacy restrictions; Chrome tab audio sharing works.
• Audio-only calls and rooms are supported and planned.
• SFUs do not decrypt media; clients request stream quality based on UI rendering size.
• Encryption uses individual sender keys rotated on membership changes for security.

---

Main Speakers / Sources

• Florian – Element Call team lead, main presenter explaining Matrix RTC concepts, Element Call architecture, and technical details.
• Aaron – Senior product designer at Element, presenting the UX and design philosophy for Element Call on web and mobile.
• Teimmo – Engineer from the VIP team, demonstrating integration of Element Call widget into a Matrix client.

---

This presentation highlights how Element Call leverages Matrix’s open, federated architecture with new RTC primitives and SFU-based backends to deliver a privacy-focused, scalable conferencing solution with rich features and broad ecosystem integration.