How to Insert DVB Subtitles and DVB Teletext On the Fly During Live Output

Overview

This article describes how to insert DVB subtitles or DVB Teletext subtitles into an output stream on the fly during live playback or recording.

In this workflow, subtitles are not loaded from a static subtitle file. Instead, the application generates subtitle data dynamically, serializes it as JSON, attaches the JSON to individual video frames, and passes those frames to the writer. The writer then encodes the subtitle data into the output stream.

The main idea is:

1. Prepare subtitle data in JSON format.
2. Attach the subtitle JSON to the required frame with MFStrSet("subtitle_json", ...).
3. Pass the frame to MWriter or MFWriter.
4. Configure the writer subtitle stream with subtitle::codec, subtitle::stream_id, and subtitle metadata.
5. The writer encodes the subtitle data into the output stream as DVB subtitles or DVB Teletext subtitles.

This approach is useful for live workflows where subtitles, captions, speaker labels, warnings, score overlays, or other text events are generated while the stream is already running.

Important concepts

There are two different codec-related values in this workflow:

• The JSON field codec describes the format of the subtitle text payload attached to the frame. In the examples below, the payload uses "ass" because the subtitle text is formatted as an ASS event.
• The writer option subtitle::codec defines how the subtitle stream is encoded into the output. For DVB subtitles, use subtitle::codec='dvbsub'. For DVB Teletext subtitles, use subtitle::codec='dvbtxt'.

The subtitle JSON is attached to a frame with the subtitle_json string key:

((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);

The subtitle stream_id in the JSON should match the writer subtitle stream ID configured with subtitle::stream_id.

Subtitle JSON structure

Each subtitle event is represented as a JSON object.

{
  "stream_id": 12,
  "codec": "ass",
  "text": "1,0,Jordan,Jordan,0,0,0,,Pack my box with five dozen liquor jugs",
  "duration_sec": 1.5
}

The supported fields used by this workflow are:

Subtitle item class

It is convenient to represent each subtitle event as a class and serialize it into JSON when the subtitle should be inserted.

using System.ComponentModel;
using System.Text.Json;
using System.Text.Json.Serialization;

public class SubtitleItem
{
    [JsonPropertyName("stream_id")]
    public int StreamId { get; set; }

    [JsonPropertyName("codec")]
    public string Codec { get; set; } = "ass";

    [JsonIgnore]
    public string Text { get; set; } = string.Empty;

    [JsonIgnore]
    public string Style { get; set; } = string.Empty;

    [EditorBrowsable(EditorBrowsableState.Never)]
    [JsonPropertyName("text")]
    public string SerializedText => Style + Text;

    [JsonPropertyName("duration_sec")]
    public double DurationSec { get; set; }

    [JsonPropertyName("ass_header")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public string? AssHeader { get; set; }

    public string ToJson(bool indented = false)
    {
        var options = new JsonSerializerOptions
        {
            WriteIndented = indented,
            DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
        };

        return JsonSerializer.Serialize(this, options);
    }
}

Prepare subtitle events

The first subtitle event can include the ASS header. The following events can reuse the same header and send only the ASS event data.

List<SubtitleItem> subtitles = new List<SubtitleItem>
{
    new SubtitleItem
    {
        StreamId = 12,
        Codec = "ass",
        AssHeader =
@"[Script Info]
Title: Live DVB Subtitle Insertion
ScriptType: v4.00+
WrapStyle: 0
ScaledBorderAndShadow: yes
YCbCr Matrix: TV.709
PlayResX: 1920
PlayResY: 1080

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Arial,58,&H00FFFFFF,&H000000FF,&H00000000,&H80000000,0,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1
Style: Alex,Arial,58,&H0000A5FF,&H000000FF,&H000000FF,&H80000000,1,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1
Style: Jordan,Arial,58,&H0000FF00,&H000000FF,&H00000000,&H80000000,1,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text",
        Style = "0,0,Alex,Alex,0,0,0,,",
        Text = "The quick brown fox jumps over the lazy dog",
        DurationSec = 1.5
    },
    new SubtitleItem
    {
        StreamId = 12,
        Codec = "ass",
        Style = "1,0,Jordan,Jordan,0,0,0,,",
        Text = "Pack my box with five dozen liquor jugs",
        DurationSec = 1.5
    },
    new SubtitleItem
    {
        StreamId = 12,
        Codec = "ass",
        Style = "2,0,Alex,Alex,0,0,0,,",
        Text = "Sphinx of black quartz, judge my vow",
        DurationSec = 1.5
    }
};

The Style value is combined with Text and serialized into the JSON text field. This allows the application to select ASS styles dynamically for different speakers or subtitle types.

Writer configuration

The writer configuration defines how the subtitle packets are encoded into the output stream.

DVB subtitle output

Use subtitle::codec='dvbsub' to encode bitmap DVB subtitles.

format='mpegts'
video::codec='libopenh264'
audio::codec='aac'
muxrate='18.5M'
video::bitrate='15M'
video::maxrate='15M'
subtitle::codec='dvbsub'
subtitle::stream_id=12
subtitle::metadata::language='eng'
program::title='FIRST TITLE'

DVB Teletext subtitle output

Use subtitle::codec='dvbtxt' to encode DVB Teletext subtitles.

format='mpegts'
video::codec='libopenh264'
audio::codec='aac'
muxrate='18.5M'
video::bitrate='15M'
video::maxrate='15M'
subtitle::codec='dvbtxt'
subtitle::stream_id=12
subtitle::metadata::language='eng'
program::title='FIRST TITLE'

The important writer subtitle options are:

MFormats SDK workflow

In MFormats SDK, read frames from MFReader, attach subtitle JSON to the required frames with MFStrSet, and pass the frames to MFWriter.

using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using MFORMATSLib;

public static class LiveSubtitleInsertionMFormats
{
    public static void Run(
        string sourcePath,
        string outputUrl,
        List<SubtitleItem> subtitles)
    {
        MFReaderClass reader = null;
        MFWriterClass writer = null;

        try
        {
            string writerConfig =
                "format='mpegts' " +
                "video::codec='libopenh264' " +
                "audio::codec='aac' " +
                "muxrate='18.5M' " +
                "video::bitrate='15M' " +
                "video::maxrate='15M' " +
                "subtitle::codec='dvbsub' " +
                "subtitle::stream_id=12 " +
                "subtitle::metadata::language='eng' " +
                "program::title='FIRST TITLE'";

            reader = new MFReaderClass();
            writer = new MFWriterClass();

            reader.ReaderOpen(sourcePath, "");
            writer.WriterSet(outputUrl, 1, writerConfig);

            int frameIndex = 0;
            int subtitleIndex = 0;

            while (true)
            {
                MFFrame frame = null;

                try
                {
                    reader.SourceFrameGet(-1, out frame, "");

                    if (frame == null)
                        break;

                    // In this example, a new subtitle is inserted every 250 frames.
                    if (frameIndex % 250 == 0 && subtitleIndex < subtitles.Count)
                    {
                        string subtitleJson = subtitles[subtitleIndex].ToJson();

                        ((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);

                        subtitleIndex++;
                    }

                    writer.ReceiverFramePut(frame, -1, "");
                }
                finally
                {
                    if (frame != null)
                        Marshal.ReleaseComObject(frame);
                }

                frameIndex++;
            }

            reader.ReaderClose();
        }
        finally
        {
            if (writer != null)
                Marshal.ReleaseComObject(writer);

            if (reader != null)
                Marshal.ReleaseComObject(reader);
        }
    }
}

The frame index controls when each subtitle appears. The subtitle duration is defined by the duration_sec value in the JSON.

MPlatform SDK workflow

In MPlatform SDK, subtitles can be inserted into frames received from the OnFrameSafe event. Enable frame data events, attach subtitle JSON to the selected frames, and start MWriter from the source object.

using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using MPLATFORMLib;

public static class LiveSubtitleInsertionMPlatform
{
    public static void Run(
        string sourcePath,
        string outputUrl,
        List<SubtitleItem> subtitles)
    {
        MFileClass file = null;
        MWriterClass writer = null;

        try
        {
            string writerConfig =
                "format='mpegts' " +
                "video::codec='libopenh264' " +
                "audio::codec='aac' " +
                "muxrate='18.5M' " +
                "video::bitrate='15M' " +
                "video::maxrate='15M' " +
                "subtitle::codec='dvbsub' " +
                "subtitle::stream_id=12 " +
                "subtitle::metadata::language='eng' " +
                "program::title='FIRST TITLE'";

            file = new MFileClass();
            writer = new MWriterClass();

            file.FileNameSet(sourcePath, "");

            file.PropsSet("object::on_frame.sync", "true");
            file.PropsSet("object::on_frame.data", "true");
            file.PropsSet("object::events.use_window", "false");

            int frameIndex = 0;
            int subtitleIndex = 0;

            file.OnFrameSafe += (string channelId, object frameObject) =>
            {
                try
                {
                    if (frameIndex % 250 == 0 &&
                        subtitleIndex < subtitles.Count)
                    {
                        string subtitleJson = subtitles[subtitleIndex].ToJson();

                        ((IMFFrame)frameObject).MFStrSet(
                            "subtitle_json",
                            subtitleJson);

                        subtitleIndex++;
                    }

                    frameIndex++;
                }
                finally
                {
                    if (frameObject != null)
                        Marshal.ReleaseComObject(frameObject);
                }
            };

            writer.WriterNameSet(outputUrl, writerConfig);

            writer.ObjectStart(file);
            file.FilePlayStart();

            // Keep the application running while the live source is being processed.
        }
        finally
        {
            if (writer != null)
                Marshal.ReleaseComObject(writer);

            if (file != null)
                Marshal.ReleaseComObject(file);
        }
    }
}

The object::on_frame.sync, object::on_frame.data, and object::events.use_window properties are used so the application receives frame objects and can attach subtitle data before the writer encodes the output.

Subtitle timing model

The subtitle appears at the time of the frame where subtitle_json is attached. The display duration is controlled by duration_sec.

For example, if a subtitle JSON object with "duration_sec": 1.5 is attached to frame 250, the subtitle starts at frame 250 and stays active for 1.5 seconds.

The examples above use a simple frame counter:

if (frameIndex % 250 == 0 && subtitleIndex < subtitles.Count)
{
    string subtitleJson = subtitles[subtitleIndex].ToJson();
    ((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);
    subtitleIndex++;
}

In a real live application, the insertion condition can be based on wall-clock time, SCTE/event metadata, operator actions, automation commands, or any other application-specific trigger.

Multiple subtitle streams

To insert several subtitle streams, use different stream IDs and configure corresponding writer subtitle tracks.

For example, one subtitle stream can use stream_id=12 and another subtitle stream can use stream_id=13.

subtitle::codec='dvbsub'
subtitle::stream_id=12
subtitle::metadata::language='eng'

subtitle.1::codec='dvbtxt'
subtitle.1::stream_id=13
subtitle.1::metadata::language='deu'

The subtitle JSON for each inserted subtitle should use the matching stream_id.

{
  "stream_id": 13,
  "codec": "ass",
  "text": "0,0,Default,Default,0,0,0,,Teletext subtitle text",
  "duration_sec": 2.0
}

Notes and limitations

• Use MFStrSet("subtitle_json", ...) to attach the subtitle JSON to a frame.
• The JSON stream_id should match the writer subtitle::stream_id.
• The JSON codec describes the subtitle payload format. In the examples above, "ass" is used for ASS-formatted subtitle event data.
• The writer subtitle::codec controls the output subtitle type: dvbsub for bitmap DVB subtitles or dvbtxt for DVB Teletext subtitles.
• Send ass_header with the first subtitle event, or when the ASS style definition changes.
• duration_sec defines how long the subtitle is displayed after it is inserted into the frame.
• For live workflows, subtitle insertion can be triggered by frame number, time, operator input, automation events, or external metadata.

Summary

To insert DVB subtitles or DVB Teletext subtitles on the fly, generate subtitle events as JSON, attach each JSON event to the required frame with MFStrSet("subtitle_json", ...), and configure the writer subtitle stream with the required output codec and stream ID.

Use subtitle::codec='dvbsub' for bitmap DVB subtitles and subtitle::codec='dvbtxt' for DVB Teletext subtitles. Make sure that the JSON stream_id matches the writer subtitle::stream_id.