MSEdgeExplainers

RTCRtpReceiver Decoder State Changed and Error Events

Authors

Much of this explainer synthesizes and consolidates prior discussions and contributions from members of the WebRTC working group.

Participate

Introduction

Game streaming platforms like Xbox Cloud Gaming and Nvidia GeForce Now rely on hardware decoding in browsers to deliver low-latency, power-efficient experiences. During a stream, the decoder's state can change. The codec can be renegotiated, the receiver can fall back from hardware to software decoding, or the decoder can fail outright. Applications have no event-driven way to observe these changes and failures as they occur. The existing statistics must be polled and terminal decoder errors are not surfaced at all.

This proposal introduces two events on the receiver. The decoderstatechange event fires when the decoder's state changes. Codec changes always fire it. Hardware-to-software decoder changes fire it only while the application is capturing. In that state, the decoderImplementation and powerEfficientDecoder statistics are already available. The decodererror event fires when the decoder hits a terminal error. Together they replace inefficient polling without exposing additional fingerprinting surface.

User-Facing Problem

When the decoder fails terminally, playback freezes. The failure is not surfaced to the application, so there is no direct signal that decoding has stopped. The decoder's state can also change during a stream, for example when the codec is renegotiated. There is no event for these changes either. The only way to observe decoder state today is to poll getStats() repeatedly, which is inefficient.

A related concern is decoder fallback. When the receiver falls back from a hardware to a software decoder, end users may experience increased latency, degraded quality, and battery drain. Developers would like to detect this in real time. They previously relied on the decoderImplementation statistic. As of Chromium M110+, it is only available while the application is capturing camera or microphone input. This proposal does not change that gating. For applications that are already capturing, it surfaces the fallback through an event instead of requiring polling. That capture-based gate fits real-time communication but not cloud gaming, another use case where low latency is essential to the user experience. A future extension could broaden the gating for decoderImplementation and powerEfficientDecoder to include signals typical of a cloud gaming session, such as gamepad input, keyboard lock, pointer lock, or fullscreen.

Goals

Non-goals

User Research

Feedback from Xbox Cloud Gaming, Nvidia GeForce Now and similar partners shows:

Proposed Approach

Introduce two events on RTCRtpReceiver:

Codec changes and decoder errors are surfaced without requiring getUserMedia() permissions. The decodererror event is coarse, carrying no decoder- or device-specific detail. Changes that reveal hardware-versus-software decoding are surfaced only when exposing hardware is allowed. This condition already gates the existing decoderImplementation and powerEfficientDecoder stats, so the event reveals nothing the page cannot already read (see Privacy Considerations). This enables applications to alert users, re-negotiate codecs, and debug issues at runtime.

Event triggers

Two changes trigger the decoderstatechange event:

The decodererror event fires when the decoder hits a terminal, unrecoverable failure, for example when hardware decoding fails and no software decoder is available for the negotiated codec (such as H.265). The failure is surfaced as an EncodingError DOMException. When a fallback succeeds (a software decoder is available), the receiver keeps decoding and may fire decoderstatechange instead, subject to the gating above.

Proposed IDL

partial interface RTCRtpReceiver {
    attribute EventHandler ondecoderstatechange;
    attribute EventHandler ondecodererror;
};

interface RTCDecoderStateChangeEvent : Event {
    constructor(DOMString type, RTCDecoderStateChangeEventInit eventInitDict);

    // The RTP timestamp of the media frame associated with this event.
    readonly attribute unsigned long rtpTimestamp;
};

interface RTCDecoderErrorEvent : RTCDecoderStateChangeEvent {
    constructor(DOMString type, RTCDecoderErrorEventInit eventInitDict);

    // The inherited rtpTimestamp reports when the error occurred.
    readonly attribute DOMException error;
};

Example

const pc = new RTCPeerConnection();

pc.addEventListener('track', (event) => {
  const receiver = event.receiver;

  // The change event fires whenever the receiver's decoder state changes.
  receiver.addEventListener('decoderstatechange', async (ev) => {
    // Query getStats() for the codec currently in use on this receiver.
    const stats = await receiver.getStats();
    let codec = 'unknown';
    for (const report of stats.values()) {
      if (report.type === 'inbound-rtp' && report.codecId) {
        const codecStats = stats.get(report.codecId);
        if (codecStats) {
          codec = `${codecStats.mimeType}|${codecStats.sdpFmtpLine}`;
        }

        // decoderImplementation and powerEfficientDecoder are permission-gated.
        // They are only present when the web app is actively capturing
        // microphone or camera input. Apps that already hold the permission can
        // still read them here.
        if (report.decoderImplementation) {
          logMetric(`Decoder implementation: ${report.decoderImplementation}`);
        }
        if (report.powerEfficientDecoder !== undefined) {
          logMetric(`Power efficient decoder: ${report.powerEfficientDecoder}`);
        }
        break;
      }
    }
    logMetric(`Decoder state change: codec=${codec}, time=${ev.rtpTimestamp}`);
  });

  // The error event reports a decoder failure.
  receiver.addEventListener('decodererror', (ev) => {
    // ev.error is a DOMException describing the failure. The inherited
    // rtpTimestamp marks when it occurred.
    showToast('Video playback error');
    logMetric(`Decoder error: ${ev.error.name} - ${ev.error.message}, time=${ev.rtpTimestamp}`);
  });
});

Alternatives Considered

  1. Use decoderImplementation info via WebRTC Stats API
    • Rejected because it now requires getUserMedia() permissions, which are invasive and have a high failure rate.
  2. Use MediaCapabilitiesInfo.powerEfficient
    • Rejected because this is a static hint that does not update when the browser silently switches from hardware to software decoding.
  3. Guess based on decode times
    • Unreliable and has masked bugs in production.
  4. Add decoderFallback field to RTCInboundRtpStreamStats
    • Rejected because relying on stats to trigger a change felt like an anti-pattern and the recommendation was to explore an event driven solution. Additionally, there were concerns around fingerprinting.
    • WebRTC March 2023 meeting – 21 March 2023

Privacy Considerations

The events carry only the media frame's rtpTimestamp. They expose no hardware vendor, device identity, or decoder detail. Applications can read decoder state through getStats(), which applies its existing privacy protections.

Stakeholder Feedback

Last discussed in the 2025-11-13 Media WG Meeting (TPAC): Slides 110-117 & minutes

References & Acknowledgements

Many thanks for valuable feedback and advice from:

Links to past working group meetings where this has been discussed:

This site is open source. Improve this page.