WebVTT subtitles for HLS

Overview

The Video SDK can write WebVTT subtitle tracks into HLS output. This workflow allows an application to load an external subtitle file, pass subtitle packets from the reader to the writer, and generate .vtt subtitle files together with the HLS playlist.

WebVTT subtitle output is configured with:

subtitle::codec='webvtt'

This mode is typically used with format='hls'. The writer creates the HLS playlist and writes WebVTT subtitle files into the output folder.

The input subtitles can be loaded from an external subtitle file through the reader external_subtitle property. For example, an ASS subtitle file can be used as the input subtitle source and converted to WebVTT in the HLS output.

Workflow

The WebVTT subtitle insertion workflow consists of the following steps:

1. Prepare an input media file and an external subtitle file.
2. Open the source with MFile.FileNameSet.
3. Attach the subtitle file with external_subtitle.
4. Enable subtitle packet output from the reader with experimental.out_subtitle_packets=2.
5. Configure MWriter for HLS output.
6. Set subtitle::codec='webvtt' in the writer configuration.
7. Start the writer and process the source.
8. Check the generated .vtt files in the HLS output folder.

Reader configuration

The reader must be configured to output subtitle packets from the attached subtitle file. The subtitle file is attached with external_subtitle.

string readerConfig =
    "external_process=false " +
    "subtitle_track=0 " +
    "experimental.out_subtitle_packets=2 " +
    @"external_subtitle=\\MLDISKSTATION\MLFiles\MediaTest\sub_idx_examples\colored-speakers.ass";

The options are:

• external_process=false — disables external processing for this reader path.
• subtitle_track=0 — selects the subtitle track.
• experimental.out_subtitle_packets=2 — enables subtitle packet output from the reader so the writer can receive and encode the subtitle stream.
• external_subtitle=... — attaches an external subtitle file to the source.

In the tested workflow, the external subtitle file is an ASS file:

colored-speakers.ass

The writer converts the subtitle packets from this source into WebVTT subtitle output for HLS.

Writer configuration

To generate HLS with WebVTT subtitles, configure MWriter with format='hls' and subtitle::codec='webvtt'.

format='hls'
mux_buffers=0
hls_time=2
hls_list_size=0
hls_playlist_type='vod'
subtitle::codec='webvtt'
subtitle::metadata::language='eng'
video::codec='libopenh264'
video::maxrate='5M'
audio::codec='aac'
program::title='FIRST TITLE'

The main options are:

• format='hls' — enables HLS output.
• hls_time=2 — sets the HLS segment duration.
• hls_list_size=0 — keeps all playlist entries.
• hls_playlist_type='vod' — creates a VOD-style HLS playlist.
• subtitle::codec='webvtt' — writes the subtitle stream as WebVTT.
• subtitle::metadata::language='eng' — sets the subtitle language metadata.
• video::codec='libopenh264' — encodes video with OpenH264.
• audio::codec='aac' — encodes audio as AAC.

MPlatform example

The following example opens a source file, attaches an external ASS subtitle file, writes HLS output with WebVTT subtitles, and then checks the generated .vtt files.

using System;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using MPLATFORMLib;

public static class WebVttHlsExample
{
    public static void WriteWebVttSubtitlesToHls()
    {
        string writerDestination = Path.Combine(
            Path.GetTempPath(),
            "MP_4180_HLS");

        string outputPlaylist = Path.Combine(writerDestination, "output.m3u8");

        string sourcePath =
            @"\\MLDISKSTATION\MLFiles\MediaTest\sub_idx_examples\input.ts";

        string subtitlePath =
            @"\\MLDISKSTATION\MLFiles\MediaTest\sub_idx_examples\colored-speakers.ass";

        string readerConfig =
            "external_process=false " +
            "subtitle_track=0 " +
            "experimental.out_subtitle_packets=2 " +
            $"external_subtitle={subtitlePath}";

        string writerConfig =
            "format='hls' " +
            "mux_buffers=0 " +
            "hls_time=2 " +
            "hls_list_size=0 " +
            "hls_playlist_type='vod' " +
            "subtitle::codec='webvtt' " +
            "subtitle::metadata::language='eng' " +
            "video::codec='libopenh264' " +
            "video::maxrate='5M' " +
            "audio::codec='aac' " +
            "program::title='FIRST TITLE'";

        MFileClass file = null;
        MWriterClass writer = null;

        try
        {
            if (Directory.Exists(writerDestination))
                Directory.Delete(writerDestination, true);

            Directory.CreateDirectory(writerDestination);

            file = new MFileClass();
            writer = new MWriterClass();

            file.FileNameSet(sourcePath, readerConfig);
            file.FilePlayStart();

            file.ObjectStateGet(out eMState fileState);

            if (fileState != eMState.eMS_Running)
                throw new InvalidOperationException("Source file did not start.");

            writer.WriterNameSet(outputPlaylist, writerConfig);
            writer.ObjectStart(file);

            writer.ObjectStateGet(out eMState writerState);

            if (writerState != eMState.eMS_Running)
                throw new InvalidOperationException("MWriter did not start.");

            // Let the writer process the source and generate HLS output.
            Thread.Sleep(15000);

            writer.ObjectClose();
            file.ObjectClose();

            string[] vttFiles = Directory.GetFiles(writerDestination, "*.vtt");

            StringBuilder vttContent = new StringBuilder();

            foreach (string currentFile in vttFiles)
            {
                string currentContent = File.ReadAllText(
                    currentFile,
                    Encoding.UTF8);

                vttContent.AppendLine(currentContent);
            }

            Console.WriteLine(vttContent.ToString());
        }
        finally
        {
            if (writer != null)
                Marshal.ReleaseComObject(writer);

            if (file != null)
                Marshal.ReleaseComObject(file);
        }
    }
}

Checking the result

After writing is complete, the HLS output folder should contain the generated playlist and WebVTT subtitle files.

The generated playlist is written to:

output.m3u8

The generated WebVTT subtitle files use the .vtt extension. Applications can check them by reading all .vtt files from the output folder:

string[] vttFiles = Directory.GetFiles(writerDestination, "*.vtt");

StringBuilder vttContent = new StringBuilder();

foreach (string currentFile in vttFiles)
{
    string currentContent = File.ReadAllText(currentFile, Encoding.UTF8);
    vttContent.AppendLine(currentContent);
}

string result = vttContent.ToString();

Console.WriteLine(result);

If the subtitle text from the input subtitle file is present in the generated WebVTT files, the subtitle insertion workflow is working correctly.

Notes and limitations

• WebVTT subtitle output is configured with subtitle::codec='webvtt'.
• WebVTT output is intended for HLS workflows and is used with format='hls'.
• The input subtitle file can be attached through external_subtitle.
• experimental.out_subtitle_packets=2 is required in this workflow so the reader outputs subtitle packets for the writer.
• ASS input subtitles can be used as the source subtitle file. The resulting HLS subtitle output is written as WebVTT.
• The generated .vtt files are written into the HLS output folder.

Summary

To write WebVTT subtitles into HLS output, attach the input subtitle file to the reader with external_subtitle, enable subtitle packet output with experimental.out_subtitle_packets=2, and configure the writer with format='hls' and subtitle::codec='webvtt'.