MSEdgeExplainers

Audio-only access UX flows

These examples illustrate possible user experiences for either proposed API. The exact UI is controlled by the browser.

Option 1: audio-only `getDisplayMedia()`

const playbackStream =
    await navigator.mediaDevices.getDisplayMedia({
      video: false,
      audio: true,
    });

This option adapts the existing display-capture entry point and picker to return only audio.

Option 2: `getPlaybackMedia()`

const playbackStream =
    await navigator.mediaDevices.getPlaybackMedia();

This option introduces a dedicated playback-capture entry point while reusing the same browser-controlled picker model.

User-selected source flow

+---------------------------------------------------+
| WEBSITE UI                                        |
|                                                   |
| Choose audio for meeting notes                    |
|                                                   |
| Select the tab or application where the meeting   |
| is playing. Only audio will be shared.             |
|                                                   |
|                         [Choose audio source]      |
+--------------------------+------------------------+
                           |
                           v
        Website calls the selected capture API
                           |
                           v
+---------------------------------------------------+
| BROWSER UI                                        |
|                                                   |
| Choose audio to share with notes.example          |
|                                                   |
| Browser tabs                                      |
|  ( ) Meeting app                                  |
|  ( ) Lecture livestream                           |
|                                                   |
| Applications                                      |
|  ( ) Microsoft Teams                              |
|  ( ) Media Player                                 |
|                                                   |
| Device audio                                      |
|  ( ) All audio playing on this device             |
|                                                   |
|                         [Cancel]  [Share audio]    |
+--------------------------+------------------------+
                           |
                           v
              Selected playback audio track
                           |
                           v
          Browser shows an ongoing indicator

The website cannot see the available sources or silently select an application. The browser owns the picker, and the user makes the final choice.

Direct system-audio flow

Some applications specifically need combined device playback rather than a single tab or application.

The website explains why system audio is required.
The user clicks a control such as Enable enhanced processing.
The browser explains that capture may include audio from multiple tabs, applications, notifications, and accessibility tools.
The user allows or denies the request.
If allowed, the browser returns an audio-only system-audio track and shows ongoing capture UI.

Illustrative browser UI:

+---------------------------------------------------+
| example.com wants to capture all audio playing   |
| on this device. This may include audio from      |
| multiple applications, websites, notifications, |
| and accessibility tools.                        |
|                                                  |
|                         [Don't allow]  [Allow]   |
+---------------------------------------------------+

This flow remains an open design option. The browser must not allow silent system-audio capture.

The picker concept is illustrative. The key requirements are:

Clearly state that only audio will be shared.
Help the user recognize which audio source they are selecting.
Represent applications that may own several windows.
Explain that system audio may include multiple applications, notifications, and audio that starts playing later.
Make narrower sources easy to choose when they satisfy the request.

Use with a microphone

A single website control can coordinate microphone and playback capture, but the playback request must start directly from the user action.

startButton.addEventListener("click", async () => {
  const playbackStream =
      await navigator.mediaDevices.getPlaybackMedia({
        audio: true,
      });

  const microphoneStream =
      await navigator.mediaDevices.getUserMedia({
        audio: true,
      });

  startFeature({
    playbackStream,
    microphoneStream,
  });
});

The playback request above could instead use audio-only getDisplayMedia():

startButton.addEventListener("click", async () => {
  const playbackStream =
      await navigator.mediaDevices.getDisplayMedia({
        video: false,
        audio: true,
      });

  const microphoneStream =
      await navigator.mediaDevices.getUserMedia({
        audio: true,
      });

  startFeature({
    playbackStream,
    microphoneStream,
  });
});

+---------------------------------------------------+
| WEBSITE UI                                        |
|                                                   |
| Create notes from the meeting audio and your      |
| microphone.                                       |
|                                                   |
|                                          [Start]  |
+--------------------------+------------------------+
                           |
                      User clicks
                           |
                           v
              Website requests playback audio
                           |
                           v
+---------------------------------------------------+
| BROWSER UI                                        |
|                                                   |
| Choose the audio source to share                  |
|                                                   |
|                               [Cancel]  [Share]   |
+--------------------------+------------------------+
                           |
                    Source selected
                           |
                           v
               Playback audio track returned
                           |
                           v
                Website requests microphone
                           |
                           v
+---------------------------------------------------+
| BROWSER UI                                        |
|                                                   |
| Allow example.com to use your microphone?         |
|                                                   |
|                         [Don't allow]  [Allow]    |
+--------------------------+------------------------+
                           |
                           v
              Website starts with available tracks

Microphone capture does not have the same transient-activation requirement, so it can be requested after the playback picker completes. The simplified examples above assume that both tracks are needed. An application that can continue with either track can handle each rejection independently or use separate controls:

[Share meeting audio]    [Use microphone]

If playback capture is declined, the application can still request and use the microphone. If microphone access is declined, it can continue using the playback track.

Communication and AI voice applications could use the playback track locally as a reference for echo cancellation and voice processing.

Ongoing controls

All successful flows end with browser-controlled ongoing-use UI:

Capture begins
      |
      v
Browser indicates which site is capturing playback audio
      |
      v
User can inspect the source and stop capture

See the ongoing capture UX mockups for additional illustrative ideas.

Playback capture ends when the user stops it, permission is revoked, the source becomes unavailable, or the website stops the track.

This site is open source. Improve this page.