Overview
This article describes how to insert DVB subtitles or DVB Teletext subtitles into an output stream on the fly during live playback or recording.
In this workflow, subtitles are not loaded from a static subtitle file. Instead, the application generates subtitle data dynamically, serializes it as JSON, attaches the JSON to individual video frames, and passes those frames to the writer. The writer then encodes the subtitle data into the output stream.
The main idea is:
- Prepare subtitle data in JSON format.
- Attach the subtitle JSON to the required frame with
MFStrSet("subtitle_json", ...). - Pass the frame to
MWriterorMFWriter. - Configure the writer subtitle stream with
subtitle::codec,subtitle::stream_id, and subtitle metadata. - The writer encodes the subtitle data into the output stream as DVB subtitles or DVB Teletext subtitles.
This approach is useful for live workflows where subtitles, captions, speaker labels, warnings, score overlays, or other text events are generated while the stream is already running.
Important concepts
There are two different codec-related values in this workflow:
- The JSON field
codecdescribes the format of the subtitle text payload attached to the frame. In the examples below, the payload uses"ass"because the subtitle text is formatted as an ASS event. - The writer option
subtitle::codecdefines how the subtitle stream is encoded into the output. For DVB subtitles, usesubtitle::codec='dvbsub'. For DVB Teletext subtitles, usesubtitle::codec='dvbtxt'.
The subtitle JSON is attached to a frame with the subtitle_json string key:
((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);The subtitle stream_id in the JSON should match the writer subtitle stream ID configured with subtitle::stream_id.
Subtitle JSON structure
Each subtitle event is represented as a JSON object.
{
"stream_id": 12,
"codec": "ass",
"text": "1,0,Jordan,Jordan,0,0,0,,Pack my box with five dozen liquor jugs",
"duration_sec": 1.5
}The supported fields used by this workflow are:
| Field | Type | Description |
|---|---|---|
stream_id | int | Subtitle stream ID. This value should match the writer option subtitle::stream_id. |
codec | string | Subtitle payload format. In this workflow, use "ass" when the subtitle text is provided as ASS event data. |
text | string | Serialized subtitle text. For ASS subtitles, this is usually an ASS event line without the full file header. |
duration_sec | double | Subtitle display duration in seconds, starting from the frame where the subtitle JSON is attached. |
ass_header | string | Optional ASS header with script, style, and event format definitions. It should be sent with the first subtitle event, or whenever the style definition needs to be changed. |
Subtitle item class
It is convenient to represent each subtitle event as a class and serialize it into JSON when the subtitle should be inserted.
using System.ComponentModel;
using System.Text.Json;
using System.Text.Json.Serialization;
public class SubtitleItem
{
[JsonPropertyName("stream_id")]
public int StreamId { get; set; }
[JsonPropertyName("codec")]
public string Codec { get; set; } = "ass";
[JsonIgnore]
public string Text { get; set; } = string.Empty;
[JsonIgnore]
public string Style { get; set; } = string.Empty;
[EditorBrowsable(EditorBrowsableState.Never)]
[JsonPropertyName("text")]
public string SerializedText => Style + Text;
[JsonPropertyName("duration_sec")]
public double DurationSec { get; set; }
[JsonPropertyName("ass_header")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? AssHeader { get; set; }
public string ToJson(bool indented = false)
{
var options = new JsonSerializerOptions
{
WriteIndented = indented,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
};
return JsonSerializer.Serialize(this, options);
}
}Prepare subtitle events
The first subtitle event can include the ASS header. The following events can reuse the same header and send only the ASS event data.
List<SubtitleItem> subtitles = new List<SubtitleItem>
{
new SubtitleItem
{
StreamId = 12,
Codec = "ass",
AssHeader =
@"[Script Info]
Title: Live DVB Subtitle Insertion
ScriptType: v4.00+
WrapStyle: 0
ScaledBorderAndShadow: yes
YCbCr Matrix: TV.709
PlayResX: 1920
PlayResY: 1080
[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Arial,58,&H00FFFFFF,&H000000FF,&H00000000,&H80000000,0,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1
Style: Alex,Arial,58,&H0000A5FF,&H000000FF,&H000000FF,&H80000000,1,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1
Style: Jordan,Arial,58,&H0000FF00,&H000000FF,&H00000000,&H80000000,1,0,0,0,100,100,0,0,1,3,1,2,10,10,40,1
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text",
Style = "0,0,Alex,Alex,0,0,0,,",
Text = "The quick brown fox jumps over the lazy dog",
DurationSec = 1.5
},
new SubtitleItem
{
StreamId = 12,
Codec = "ass",
Style = "1,0,Jordan,Jordan,0,0,0,,",
Text = "Pack my box with five dozen liquor jugs",
DurationSec = 1.5
},
new SubtitleItem
{
StreamId = 12,
Codec = "ass",
Style = "2,0,Alex,Alex,0,0,0,,",
Text = "Sphinx of black quartz, judge my vow",
DurationSec = 1.5
}
};The Style value is combined with Text and serialized into the JSON text field. This allows the application to select ASS styles dynamically for different speakers or subtitle types.
Writer configuration
The writer configuration defines how the subtitle packets are encoded into the output stream.
DVB subtitle output
Use subtitle::codec='dvbsub' to encode bitmap DVB subtitles.
format='mpegts'
video::codec='libopenh264'
audio::codec='aac'
muxrate='18.5M'
video::bitrate='15M'
video::maxrate='15M'
subtitle::codec='dvbsub'
subtitle::stream_id=12
subtitle::metadata::language='eng'
program::title='FIRST TITLE'DVB Teletext subtitle output
Use subtitle::codec='dvbtxt' to encode DVB Teletext subtitles.
format='mpegts'
video::codec='libopenh264'
audio::codec='aac'
muxrate='18.5M'
video::bitrate='15M'
video::maxrate='15M'
subtitle::codec='dvbtxt'
subtitle::stream_id=12
subtitle::metadata::language='eng'
program::title='FIRST TITLE'The important writer subtitle options are:
| Option | Description |
|---|---|
subtitle::codec | Output subtitle codec. Use dvbsub for bitmap DVB subtitles or dvbtxt for DVB Teletext subtitles. |
subtitle::stream_id | Subtitle stream ID in the output. This value should match the stream_id value in the subtitle JSON. |
subtitle::metadata::language | Subtitle language metadata, for example eng. |
program::title | Program title metadata for the output transport stream. |
MFormats SDK workflow
In MFormats SDK, read frames from MFReader, attach subtitle JSON to the required frames with MFStrSet, and pass the frames to MFWriter.
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using MFORMATSLib;
public static class LiveSubtitleInsertionMFormats
{
public static void Run(
string sourcePath,
string outputUrl,
List<SubtitleItem> subtitles)
{
MFReaderClass reader = null;
MFWriterClass writer = null;
try
{
string writerConfig =
"format='mpegts' " +
"video::codec='libopenh264' " +
"audio::codec='aac' " +
"muxrate='18.5M' " +
"video::bitrate='15M' " +
"video::maxrate='15M' " +
"subtitle::codec='dvbsub' " +
"subtitle::stream_id=12 " +
"subtitle::metadata::language='eng' " +
"program::title='FIRST TITLE'";
reader = new MFReaderClass();
writer = new MFWriterClass();
reader.ReaderOpen(sourcePath, "");
writer.WriterSet(outputUrl, 1, writerConfig);
int frameIndex = 0;
int subtitleIndex = 0;
while (true)
{
MFFrame frame = null;
try
{
reader.SourceFrameGet(-1, out frame, "");
if (frame == null)
break;
// In this example, a new subtitle is inserted every 250 frames.
if (frameIndex % 250 == 0 && subtitleIndex < subtitles.Count)
{
string subtitleJson = subtitles[subtitleIndex].ToJson();
((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);
subtitleIndex++;
}
writer.ReceiverFramePut(frame, -1, "");
}
finally
{
if (frame != null)
Marshal.ReleaseComObject(frame);
}
frameIndex++;
}
reader.ReaderClose();
}
finally
{
if (writer != null)
Marshal.ReleaseComObject(writer);
if (reader != null)
Marshal.ReleaseComObject(reader);
}
}
}The frame index controls when each subtitle appears. The subtitle duration is defined by the duration_sec value in the JSON.
MPlatform SDK workflow
In MPlatform SDK, subtitles can be inserted into frames received from the OnFrameSafe event. Enable frame data events, attach subtitle JSON to the selected frames, and start MWriter from the source object.
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using MPLATFORMLib;
public static class LiveSubtitleInsertionMPlatform
{
public static void Run(
string sourcePath,
string outputUrl,
List<SubtitleItem> subtitles)
{
MFileClass file = null;
MWriterClass writer = null;
try
{
string writerConfig =
"format='mpegts' " +
"video::codec='libopenh264' " +
"audio::codec='aac' " +
"muxrate='18.5M' " +
"video::bitrate='15M' " +
"video::maxrate='15M' " +
"subtitle::codec='dvbsub' " +
"subtitle::stream_id=12 " +
"subtitle::metadata::language='eng' " +
"program::title='FIRST TITLE'";
file = new MFileClass();
writer = new MWriterClass();
file.FileNameSet(sourcePath, "");
file.PropsSet("object::on_frame.sync", "true");
file.PropsSet("object::on_frame.data", "true");
file.PropsSet("object::events.use_window", "false");
int frameIndex = 0;
int subtitleIndex = 0;
file.OnFrameSafe += (string channelId, object frameObject) =>
{
try
{
if (frameIndex % 250 == 0 &&
subtitleIndex < subtitles.Count)
{
string subtitleJson = subtitles[subtitleIndex].ToJson();
((IMFFrame)frameObject).MFStrSet(
"subtitle_json",
subtitleJson);
subtitleIndex++;
}
frameIndex++;
}
finally
{
if (frameObject != null)
Marshal.ReleaseComObject(frameObject);
}
};
writer.WriterNameSet(outputUrl, writerConfig);
writer.ObjectStart(file);
file.FilePlayStart();
// Keep the application running while the live source is being processed.
}
finally
{
if (writer != null)
Marshal.ReleaseComObject(writer);
if (file != null)
Marshal.ReleaseComObject(file);
}
}
}The object::on_frame.sync, object::on_frame.data, and object::events.use_window properties are used so the application receives frame objects and can attach subtitle data before the writer encodes the output.
Subtitle timing model
The subtitle appears at the time of the frame where subtitle_json is attached. The display duration is controlled by duration_sec.
For example, if a subtitle JSON object with "duration_sec": 1.5 is attached to frame 250, the subtitle starts at frame 250 and stays active for 1.5 seconds.
The examples above use a simple frame counter:
if (frameIndex % 250 == 0 && subtitleIndex < subtitles.Count)
{
string subtitleJson = subtitles[subtitleIndex].ToJson();
((IMFFrame)frame).MFStrSet("subtitle_json", subtitleJson);
subtitleIndex++;
}In a real live application, the insertion condition can be based on wall-clock time, SCTE/event metadata, operator actions, automation commands, or any other application-specific trigger.
Multiple subtitle streams
To insert several subtitle streams, use different stream IDs and configure corresponding writer subtitle tracks.
For example, one subtitle stream can use stream_id=12 and another subtitle stream can use stream_id=13.
subtitle::codec='dvbsub'
subtitle::stream_id=12
subtitle::metadata::language='eng'
subtitle.1::codec='dvbtxt'
subtitle.1::stream_id=13
subtitle.1::metadata::language='deu'The subtitle JSON for each inserted subtitle should use the matching stream_id.
{
"stream_id": 13,
"codec": "ass",
"text": "0,0,Default,Default,0,0,0,,Teletext subtitle text",
"duration_sec": 2.0
}Notes and limitations
- Use
MFStrSet("subtitle_json", ...)to attach the subtitle JSON to a frame. - The JSON
stream_idshould match the writersubtitle::stream_id. - The JSON
codecdescribes the subtitle payload format. In the examples above,"ass"is used for ASS-formatted subtitle event data. - The writer
subtitle::codeccontrols the output subtitle type:dvbsubfor bitmap DVB subtitles ordvbtxtfor DVB Teletext subtitles. - Send
ass_headerwith the first subtitle event, or when the ASS style definition changes. duration_secdefines how long the subtitle is displayed after it is inserted into the frame.- For live workflows, subtitle insertion can be triggered by frame number, time, operator input, automation events, or external metadata.
Summary
To insert DVB subtitles or DVB Teletext subtitles on the fly, generate subtitle events as JSON, attach each JSON event to the required frame with MFStrSet("subtitle_json", ...), and configure the writer subtitle stream with the required output codec and stream ID.
Use subtitle::codec='dvbsub' for bitmap DVB subtitles and subtitle::codec='dvbtxt' for DVB Teletext subtitles. Make sure that the JSON stream_id matches the writer subtitle::stream_id.