Insert DVB subtitles or DVB Teletext during live output

Overview

Use this workflow when an operator or application produces subtitle text while the output is running. Attach each event to a video frame as subtitle_json, then send the frame to the writer. The frame timestamp determines when the event starts; duration_sec controls how long it is displayed.

The JSON payload format and the writer output codec are independent. The sample sends ASS event data; the writer can encode it as bitmap DVB subtitles (dvbsub) or DVB Teletext (dvbtxt). Use an x64 application for frame-level subtitle insertion.

Quick start: Subtitle Generation Sample

Open the x64 shortcut from your installed SDK, or open the solution in the sample folder.

1. In Input Source, select a source type. For a first run, choose File / URL, enter the source, and select Play.
2. In Subtitle Queue, configure the track's stream ID, PID, language, and DVB type. Use the supplied cues, edit them, or load cues into the queue.
3. In Output, select the destination and select Start Output. Leave Start subtitles with output checked to send the queue automatically, or uncheck it for a manual test.
4. To send one event, select the target under Track / PID in Live JSON Injection, review the payload, and select Send Now. To start the prepared queue manually, select Send from Queue.
5. Check the result at the receiver using the MPEG-TS playback guide.

Use this sample to test event insertion. To attach existing ASS or SRT files directly at the reader, follow the Conversion workflow.

Match the JSON event to the writer track

There are two different codec-related values in this workflow:

• The JSON field codec describes the format of the subtitle text payload attached to the frame. In the examples below, the payload uses "ass" because the subtitle text is formatted as an ASS event.
• The writer option subtitle::codec defines how the subtitle stream is encoded into the output. For DVB subtitles, use subtitle::codec='dvbsub'. For DVB Teletext subtitles, use subtitle::codec='dvbtxt'.

The subtitle JSON is attached to a frame with the subtitle_json string key:

((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);

The subtitle stream_id in the JSON should match the writer subtitle stream ID configured with subtitle::stream_id.

The JSON stream_id and the frame property name serve different purposes. stream_id identifies the SDK subtitle stream, while the frame property name selects the writer track slot. Use subtitle_json for subtitle::..., subtitle_json.1 for subtitle.1::..., and so on.

To set the PID visible in the transport stream, use the matching writer track's pid option, such as subtitle::pid=117. It does not replace the JSON stream_id or select the frame property index.

Subtitle JSON structure

Each subtitle event is represented as a JSON object.

{
  "stream_id": 12,
  "codec": "ass",
  "text": "1,0,Jordan,Jordan,0,0,0,,Pack my box with five dozen liquor jugs",
  "duration_sec": 1.5
}

The supported fields used by this workflow are:

Subtitle item class

It is convenient to represent each subtitle event as a class and serialize it into JSON when the subtitle should be inserted.

using System.ComponentModel;
using System.Text.Json;
using System.Text.Json.Serialization;

public class SubtitleItem
{
    [JsonPropertyName("stream_id")]
    public int StreamId { get; set; }

    [JsonPropertyName("codec")]
    public string Codec { get; set; } = "ass";

    [JsonIgnore]
    public string Text { get; set; } = string.Empty;

    [JsonIgnore]
    public string Style { get; set; } = string.Empty;

    [EditorBrowsable(EditorBrowsableState.Never)]
    [JsonPropertyName("text")]
    public string SerializedText => Style + Text;

    [JsonPropertyName("duration_sec")]
    public double DurationSec { get; set; }

    [JsonPropertyName("ass_header")]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public string? AssHeader { get; set; }

    public string ToJson(bool indented = false)
    {
        var options = new JsonSerializerOptions
        {
            WriteIndented = indented,
            DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
        };

        return JsonSerializer.Serialize(this, options);
    }
}

Prepare subtitle events

The first subtitle event can include the ASS header. The following events can reuse the same header and send only the ASS event data.

List<SubtitleItem> subtitles = new List<SubtitleItem>
{
    new SubtitleItem
    {
        StreamId = 12,
        Codec = "ass",
        AssHeader =
@"[Script Info]
Title: Live DVB Subtitle Insertion
ScriptType: v4.00+
WrapStyle: 0
ScaledBorderAndShadow: yes
YCbCr Matrix: TV.709
PlayResX: 1920
PlayResY: 1080

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Arial,58,&H00FFFFFF,&H000000FF,&H00000000,&H80000000,0,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1
Style: Alex,Arial,58,&H0000A5FF,&H000000FF,&H000000FF,&H80000000,1,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1
Style: Jordan,Arial,58,&H0000FF00,&H000000FF,&H00000000,&H80000000,1,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text",
        Style = "0,0,Alex,Alex,0,0,0,,",
        Text = "The quick brown fox jumps over the lazy dog",
        DurationSec = 1.5
    },
    new SubtitleItem
    {
        StreamId = 12,
        Codec = "ass",
        Style = "1,0,Jordan,Jordan,0,0,0,,",
        Text = "Pack my box with five dozen liquor jugs",
        DurationSec = 1.5
    },
    new SubtitleItem
    {
        StreamId = 12,
        Codec = "ass",
        Style = "2,0,Alex,Alex,0,0,0,,",
        Text = "Sphinx of black quartz, judge my vow",
        DurationSec = 1.5
    }
};

The Style value is combined with Text and serialized into the JSON text field. This allows the application to select ASS styles dynamically for different speakers or subtitle types.

Do not send only plain text when codec is "ass". The text field must include the ASS event payload generated by Style + Text.

Escaping the subtitle text

The text of an ASS event is not plain text. A line break is written as \N, and the characters { and } start and end the ASS override tags, so they have to be escaped when the text comes from a user or from a speech-recognition service. The sample does it like this:

public static string EscapeAssText(string text)
{
    return (text ?? string.Empty)
        .Replace("\r\n", "\\N")
        .Replace("\n", "\\N")
        .Replace("\r", "\\N")
        .Replace("{", "\\{")
        .Replace("}", "\\}");
}

Plain text subtitles

When the text does not need ASS styles, send it as plain text. The JSON then contains only the stream ID, the text and, if the default of 3 seconds does not fit, the duration. There is no ass_header, so the SDK uses the srt payload format:

{
  "stream_id": 12,
  "text": "Plain subtitle text",
  "duration_sec": 2.0
}

The writer track settings (subtitle::codec, subtitle::stream_id) are the same as for ASS subtitles.

Writer configuration

The writer configuration defines how the subtitle packets are encoded into the output stream.

DVB subtitle output

Use subtitle::codec='dvbsub' to encode bitmap DVB subtitles.

format='mpegts'
video::codec='libopenh264'
audio::codec='aac'
muxrate='18.5M'
video::bitrate='15M'
video::maxrate='15M'
subtitle::codec='dvbsub'
subtitle::stream_id=12
subtitle::metadata::language='eng'
program::title='FIRST TITLE'

DVB Teletext subtitle output

Use subtitle::codec='dvbtxt' to encode DVB Teletext subtitles.

format='mpegts'
video::codec='libopenh264'
audio::codec='aac'
muxrate='18.5M'
video::bitrate='15M'
video::maxrate='15M'
subtitle::codec='dvbtxt'
subtitle::stream_id=12
subtitle::metadata::language='eng'
program::title='FIRST TITLE'

The important writer subtitle options are:

MFormats SDK workflow

In MFormats SDK, read frames from MFReader, attach subtitle JSON to the required frames with MFStrSet, and pass the frames to MFWriter.

using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using MFORMATSLib;

public static class LiveSubtitleInsertionMFormats
{
    public static void Run(
        string sourcePath,
        string outputUrl,
        List<SubtitleItem> subtitles)
    {
        MFReaderClass reader = null;
        MFWriterClass writer = null;

        try
        {
            string writerConfig =
                "format='mpegts' " +
                "video::codec='libopenh264' " +
                "audio::codec='aac' " +
                "muxrate='18.5M' " +
                "video::bitrate='15M' " +
                "video::maxrate='15M' " +
                "subtitle::codec='dvbsub' " +
                "subtitle::stream_id=12 " +
                "subtitle::metadata::language='eng' " +
                "program::title='FIRST TITLE'";

            reader = new MFReaderClass();
            writer = new MFWriterClass();

            reader.ReaderOpen(sourcePath, "");
            writer.WriterSet(outputUrl, 1, writerConfig);

            int frameIndex = 0;
            int subtitleIndex = 0;

            while (true)
            {
                MFFrame frame = null;

                try
                {
                    reader.SourceFrameGet(-1, out frame, "");

                    if (frame == null)
                        break;

                    // In this example, a new subtitle is inserted every 250 frames.
                    if (frameIndex % 250 == 0 && subtitleIndex < subtitles.Count)
                    {
                        string subtitleJson = subtitles[subtitleIndex].ToJson();

                        ((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);

                        subtitleIndex++;
                    }

                    writer.ReceiverFramePut(frame, -1, "");
                }
                finally
                {
                    if (frame != null)
                        Marshal.ReleaseComObject(frame);
                }

                frameIndex++;
            }

            reader.ReaderClose();
        }
        finally
        {
            if (writer != null)
                Marshal.ReleaseComObject(writer);

            if (reader != null)
                Marshal.ReleaseComObject(reader);
        }
    }
}

The frame index controls when each subtitle appears. The subtitle duration is defined by the duration_sec value in the JSON.

MPlatform SDK workflow

In MPlatform SDK, subtitles can be inserted into frames received from the OnFrameSafe event. Enable frame data events, attach subtitle JSON to the selected frames, and start MWriter from the source object.

using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using MPLATFORMLib;

public static class LiveSubtitleInsertionMPlatform
{
    public static void Run(
        string sourcePath,
        string outputUrl,
        List<SubtitleItem> subtitles)
    {
        MFileClass file = null;
        MWriterClass writer = null;

        try
        {
            string writerConfig =
                "format='mpegts' " +
                "video::codec='libopenh264' " +
                "audio::codec='aac' " +
                "muxrate='18.5M' " +
                "video::bitrate='15M' " +
                "video::maxrate='15M' " +
                "subtitle::codec='dvbsub' " +
                "subtitle::stream_id=12 " +
                "subtitle::metadata::language='eng' " +
                "program::title='FIRST TITLE'";

            file = new MFileClass();
            writer = new MWriterClass();

            file.FileNameSet(sourcePath, "");

            file.PropsSet("object::on_frame.sync", "true");
            file.PropsSet("object::on_frame.data", "true");
            file.PropsSet("object::events.use_window", "false");

            int frameIndex = 0;
            int subtitleIndex = 0;

            file.OnFrameSafe += (string channelId, object frameObject) =>
            {
                try
                {
                    if (frameIndex % 250 == 0 &&
                        subtitleIndex < subtitles.Count)
                    {
                        string subtitleJson = subtitles[subtitleIndex].ToJson();

                        ((IMFFrame)frameObject).MFStrSet(
                            "subtitle_json",
                            subtitleJson);

                        subtitleIndex++;
                    }

                    frameIndex++;
                }
                finally
                {
                    if (frameObject != null)
                        Marshal.ReleaseComObject(frameObject);
                }
            };

            writer.WriterNameSet(outputUrl, writerConfig);

            writer.ObjectStart(file);
            file.FilePlayStart();

            // Keep the application running while the live source is being processed.
        }
        finally
        {
            if (writer != null)
                Marshal.ReleaseComObject(writer);

            if (file != null)
                Marshal.ReleaseComObject(file);
        }
    }
}

The object::on_frame.sync, object::on_frame.data, and object::events.use_window properties are used so the application receives frame objects and can attach subtitle data before the writer encodes the output.

How live subtitle timing works

The subtitle JSON does not contain a separate start-time field. A subtitle starts at the timestamp of the frame where the application attaches subtitle_json.

((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);

The display duration is controlled by the JSON duration_sec value. In other words:

subtitle start time = timestamp of the frame where subtitle_json is attached
subtitle end time   = subtitle start time + duration_sec

For example, if a subtitle JSON object with "duration_sec": 1.5 is attached to frame 250, the subtitle starts at frame 250 and remains active for 1.5 seconds. The exact media time of frame 250 depends on the frame rate. For example, at 25 fps, frame 250 is approximately 10 seconds from the start; at 50 fps, frame 250 is approximately 5 seconds from the start.

The examples above use a simple frame counter only to demonstrate the mechanism:

if (frameIndex % 250 == 0 && subtitleIndex < subtitles.Count)
{
    string subtitleJson = subtitles[subtitleIndex].ToJson();
    ((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);
    subtitleIndex++;
}

This does not mean that subtitles can only be inserted every 250 frames. The value 250 is just a sample trigger. The application can attach subtitle_json to any frame.

In a real live workflow, the insertion condition can be based on frame PTS, wall-clock time, speech-recognition timestamps, SCTE/event metadata, operator actions, automation commands, or any other application-specific trigger.

For example, insertion can be based on the current frame timestamp:

M_TIME frameTime;
((IMFFrame)frame).MFTimeGet(out frameTime);

double frameTimeSec = frameTime.rtStartTime / 10000000.0;

if (frameTimeSec >= subtitleStartTimeSec)
{
    ((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);
}

Or the subtitle can be inserted as soon as a new live subtitle event is available:

if (TryGetNextSubtitle(out string subtitleJson))
{
    ((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);
}

For speech-recognition workflows, the application can use the transcript timestamp, find the matching buffered video frame, and attach the JSON to that specific frame:

if (frameTimeSec >= transcriptStartTimeSec)
{
    string subtitleJson = transcriptSubtitle.ToJson();
    ((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);
}

The JSON is attached once at the subtitle start frame. The duration_sec value defines how long the subtitle should remain visible after that start frame. It does not require the application to insert the same JSON on every following frame.

Multiple subtitle streams

To insert several subtitle streams, configure a writer subtitle track for each stream and use a different JSON stream_id for each track.

For example, one subtitle stream can use stream_id=12 and another subtitle stream can use stream_id=13.

subtitle::codec='dvbsub'
subtitle::stream_id=12
subtitle::metadata::language='eng'

subtitle.1::codec='dvbtxt'
subtitle.1::stream_id=13
subtitle.1::metadata::language='deu'

The subtitle JSON for each inserted subtitle must use the matching stream_id. The frame property name must also match the writer track index.

The following example inserts DVB bitmap subtitles on writer track 0 and DVB Teletext subtitles on writer track 1. Both events can be attached to the same video frame because they use different frame property names.

string dvbSubtitleJson = new SubtitleItem
{
    StreamId = 12,
    Codec = "ass",
    AssHeader = assHeader,
    Style = "0,0,Default,Default,0,0,0,,",
    Text = "DVB subtitle text",
    DurationSec = 2.0
}.ToJson();

string teletextSubtitleJson = new SubtitleItem
{
    StreamId = 13,
    Codec = "ass",
    AssHeader = assHeader,
    Style = "0,0,Default,Default,0,0,0,,",
    Text = "DVB Teletext subtitle text",
    DurationSec = 2.0
}.ToJson();

((IMFFrame)frame).MFStrSet("subtitle_json", dvbSubtitleJson);
((IMFFrame)frame).MFStrSet("subtitle_json.1", teletextSubtitleJson);

Do not attach every JSON object with subtitle_json. In that case, the writer receives all events in track 0 and the other configured subtitle tracks do not receive subtitle data.

Notes and limitations

• Inserting subtitles with subtitle_json is supported in 64-bit applications.
• The stream_id and the text are required. A JSON that cannot be parsed, has no text, or has a stream_id that no writer track uses is skipped without an error: the SDK only writes a message to its log.
• One frame carries one event per writer track. If a second event for the same stream is attached to the same frame, it is rejected. Attach it to the next frame, or use the frame property of another track.
• ass_header is remembered for the stream after the first event, so it is enough to send it once and again when the style definition changes.
• The subtitle starts at the time stamp of the frame and lasts duration_sec; there is no separate start time in the JSON.
• Use MFStrSet("subtitle_json", ...) to attach the subtitle JSON to writer track 0. For writer track subtitle.N, use MFStrSet("subtitle_json.N", ...).
• The JSON stream_id should match the writer subtitle::stream_id.
• The frame property index must match the writer track index. For example, subtitle.1::stream_id=13 receives JSON from subtitle_json.1, not from subtitle_json.
• The JSON codec describes the subtitle payload format. In the examples above, "ass" is used for ASS-formatted subtitle event data.
• The writer subtitle::codec controls the output subtitle type: dvbsub for bitmap DVB subtitles or dvbtxt for DVB Teletext subtitles.
• Send ass_header with the first subtitle event, or when the ASS style definition changes.
• duration_sec defines how long the subtitle is displayed after it is inserted into the frame.
• For live workflows, subtitle insertion can be triggered by frame number, time, operator input, automation events, or external metadata.

Summary

To insert DVB subtitles or DVB Teletext subtitles on the fly, generate subtitle events as JSON, attach each JSON event to the required frame with MFStrSet("subtitle_json", ...), and configure the writer subtitle stream with the required output codec and stream ID. For multiple subtitle tracks, use the corresponding frame property name: subtitle_json, subtitle_json.1, subtitle_json.2, and so on.

Use subtitle::codec='dvbsub' for bitmap DVB subtitles and subtitle::codec='dvbtxt' for DVB Teletext subtitles. Make sure that the JSON stream_id matches the writer subtitle::stream_id, and that ASS JSON text contains the complete ASS event payload, including the event fields before the visible text.

See also

• Subtitles in Video SDK: overview and samples
• Insert ASS or SRT subtitles into MPEG-TS
• Display DVB subtitles and DVB Teletext from MPEG-TS
• Write WebVTT subtitles for HLS