In-App Music for Live Rooms and Broadcasts

A host who plays a song through their speakers and lets the microphone pick it up sends the audience a recording of a room: thin, late, and chewed up by echo cancellation, whose whole job is to remove that sound. VoltKit RTC takes the music from inside your app instead and sends it as audio in its own right.

• The audience hears the music clean.
• Music and microphone have separate switches: turn the music off and keep talking, or mute yourself and let the song play.
• In a room, music is its own track, so each viewer can turn it down without losing the host.

In a room (browser)

const song = document.querySelector("audio") as HTMLAudioElement;      // <audio src="/song.mp3" crossorigin="anonymous">

await client.publishMusic(song, { volume: 0.8 });  // call it from a click
await song.play();

client.setMusicVolume(0.4);                        // for the audience
await client.setMusicEnabled(false);               // music off, host still talking
await client.setMicEnabled(false);                 // mic off, music still playing
await client.stopMusic();

publishMusic accepts an <audio> or <video> element, a MediaStream or a MediaStreamTrack. Only roles that may publish (host, cohost) can call it; anyone else gets forbidden.

The host keeps hearing the music while it is on air. Pass { monitor: false } to send it without playing it locally.

Hearing it

Viewers receive the music as a second audio track. source says which is which:

client.on("track", ({ identity, kind, source, track }) => {
  if (source === "music") {
    musicAudio.srcObject = new MediaStream([track]);   // its own element, its own volume
  } else {
    attach(identity, kind, track);
  }
});

client.on("trackMuted", ({ source, paused }) => {
  if (source === "music") showMusicPaused(paused);
});

musicAudio.muted = true;     // "mute the music" for this viewer only

// Flutter
client.onTrack.listen((t) {
  if (t.isMusic) musicRenderer.srcObject = t.stream;
});
client.onTrackMuted.listen((m) {
  if (m.source == 'music') setState(() => musicPlaying = !m.paused);
});

source is mic, music, camera or screen. It is also on each entry of producers in the join result, so someone who joins after the music started is told too.

On a broadcast

A broadcast carries a single audio track, so the music is mixed with the microphone in the browser before it is sent. It still never passes through the microphone.

await publisher.setMusic(song, { volume: 0.8 });   // while live, or start({ music: song })

publisher.setMusicVolume(0.4);
publisher.setMusicEnabled(false);                  // music off, mic untouched
publisher.setMicEnabled(false);                    // mic off, music keeps playing
publisher.clearMusic();

Adding music to a broadcast that is already live does not interrupt it. A broadcast is billed at the video rate whatever its audio carries, so the music never changes its rate either.

Flutter on Android and iOS

Receiving music works on every platform: RemoteTrack.source and onTrackMuted are in the Flutter SDK.

Publishing music uses the same API:

await client.publishMusic(musicStream);      // a MediaStream holding the audio
await client.setMusicEnabled(false);
await client.setMicEnabled(false);           // independent of the music
await client.stopMusic();

What differs by platform is where musicStream comes from:

On Android and iOS a clean music track therefore needs a native audio source in your app. Until it has one, mobile hosts can still be heard and can receive music published by a web host, with the separate controls above. If this matters for your launch, write to contact@voltkit.in and tell us which platform comes first.

Under the hood

Publishing a track with source: "music" is an ordinary produce on the realtime protocol, encoded for music rather than speech: stereo, 128 kbps, with silence suppression off so quiet passages are not chopped. Muting is pauseProducer, which stops the audio at the source and tells every viewer.

The track.published webhook carries source, so your backend can tell when a host starts music.