Generating custom CC608 subtitles and inserting them into ANC packets

Video SDKs provide tools for embedding custom metadata or ancillary data (ANC packets) into each frame of video sources. Once this data is added, the frames can be sent to an SDI output or recorded into a video file. One common use for ANC packets is to insert closed captioning data (EIA-608). This article explains the fundamental principles and methods for implementing this functionality in your solutions.

To achieve this goal, the following steps have to be performed:

1. Develop a logic that converts text into valid CC608 byte pairs and inserts these bytes into the ANC packets of each video frame, continuing until the entire text has been processed.

2. Pass the modified frames to MWriter/MFWriter or MRenderer/MFRenderer, depending on the scenario.

3. Depending on the scenario, receive the video with embedded CC608 subtitles. If you are receiving the video via SDK based applications via SDI, additional configuration of the MLive/MFLive properties is required.

Speaking about the first paragraph, there are four features to which attention should be paid:

1. All the CC608 commands and data (one pair of chars) inside ANC packets might be stored inside 2 bytes (16 bits). They can be stored, for instance, into Uint16 variables.

2. Apply Odd Parity to CC608 Byte Pairs.

3. The process of transmitting the CCs starts with initializing commands.

4. It's possible to send only 2 chars or 1 command of CC per 1 frame.

5. By default, empty CC608 ANC data (0x8080) must be sent every frame if required by the hardware.

More information about CC608 and their structure can be found here → https://en.wikipedia.org/wiki/EIA-608

Converting text to CC608 byte pairs

In the process of embedding text as closed captions (CC608), the text needs to be converted into a specific format of byte pairs that comply with the CC608 specification. The conversion begins by preparing the text for encoding, where each character is processed into a 2-byte pair. The bytes are then modified to ensure they meet the odd parity requirement, a crucial step for compliance with the CC608 standard. As shown in the code, each byte is examined, and the least significant bit (bit 7) is adjusted based on the parity of the other bits in the byte. This ensures that the total number of 1-bits in the byte is odd or even as required.

After the parity adjustment, the resulting byte pairs are packed into the appropriate format for transmission or embedding, with any remaining spaces padded and empty packets inserted where necessary to align with the required structure. Importantly, if there is no more text to encode, the process still requires inserting empty packets (0x8080) into each video frame to maintain the correct packet structure. This ensures that every frame, even if it contains no text, has a valid data stream. The process continues until all the text is converted into the correct byte pairs, ready to be inserted into ANC packets for video frames.

An example of the code for converting a string into CC608 byte pairs is provided below:

// Modify the CC608 byte to ensure it adheres to the CC608 specification by applying odd parity.
public Byte ApplyOddParity(Byte b)
{
    Int32 numOnes = 0;

    // Count the number of 1s in the lower 7 bits (excluding the parity bit):
    for (Int32 i = 0; i < 7; i++) // only lower 7 bits
    {
        if (((b >> i) & 1) == 1)
        {
            numOnes++;
        }
    }

    // set LastBit to 1 when number of 1 is even 
    // clear LastBit to 0 when number of 1 is odd
    if (numOnes % 2 == 0)
    {
        return (Byte)(b | 0x80);   // Set bit 7 to 1 to make the total number of 1s odd
    }
    else
    {
        return (Byte)(b & 0x7F);   // Clear bit 7 to keep the total number of 1s odd
    }
}


// Converting a String into a Collection of CC608 Byte Pairs as Uint16:
public List<UInt16> Text_To_CC608_PacketConverter(String TextToEmbedAsCC608)
{
    List<UInt16> CC_BytePacketsData = new List<UInt16>();

    Byte[] DataBytes;
    Int32 TextPos = 0;

    // CR (Carriage Return)
    CC_BytePacketsData.Add(0x2614); // RU3 Roll-Up 3 rows
    CC_BytePacketsData.Add(0x6011); // PAC (Preamble Address Code), CC1, Row 15, default style
    
    while (TextPos < TextToEmbedAsCC608.Length)
    {
        if (TextPos % 32 == 0)
        {
            CC_BytePacketsData.Add(0x2D14); // CR (Carriage Return)
        }


        // PadRight(2, '\0') ensures the string is exactly 2 characters long.
        // If the substring is shorter than 2 characters, it appends null characters ('\0')
        // to the right until the length reaches 2. If it's already 2 characters, it remains unchanged.
        DataBytes = Encoding.ASCII.GetBytes(TextToEmbedAsCC608.Substring(TextPos, Math.Min(2, TextToEmbedAsCC608.Length - TextPos)).PadRight(2, '\0'));
        
        TextPos = TextPos + 2;

        CC_BytePacketsData.Add(BitConverter.ToUInt16(DataBytes, 0));
    }


    // Insert 25 empty packets (Optional):
    for (TextPos = 0; TextPos < 25; TextPos = TextPos + 2)
    {
        CC_BytePacketsData.Add(0x8080);
    }

    // CR (Carriage Return)
    CC_BytePacketsData.Add(0x2614);
    CC_BytePacketsData.Add(0x6011);

    // Apply bit parity to ALL CC608 byte pairs (commands, data, empty packets):
    for (Int32 i = 0; i < CC_BytePacketsData.Count; i++)
    {
        DataBytes = BitConverter.GetBytes(CC_BytePacketsData[i]);

    
        DataBytes[0] = ApplyOddParity(DataBytes[0]);
        DataBytes[1] = ApplyOddParity(DataBytes[1]);

    
        CC_BytePacketsData[i] = BitConverter.ToUInt16(DataBytes, 0);
    }

    Console.WriteLine(String.Join(Environment.NewLine, CC_BytePacketsData.Select(e => "0x" + e.ToString("X4")).ToList()));

    return CC_BytePacketsData;

}

Once the text is converted into a collection of CC608 byte pairs, it's ready to be embedded into the ANC data of the frames and sent to the renderer or writer.

Embedding CC Packets into the video stream

Once the text has been converted into CC byte pairs, these packets need to be correctly embedded into the video stream. Since the MFormats SDK operates at the frame level, we have the flexibility to modify frames in any way we wish, as long as we have access to them in our code. However, if you are working with the MPlatform SDK, you will need to attach the OnFrameSafe event to the object instance and enable frame data modification. This can be done on objects such as MFile, MLive, MRenderer, or MWriter, depending on the scenario.

MPlatform SDK:

// Attaching OnFrameSafe event to the SDK object and allowing to modify data there:
m_objFile.OnFrameSafe += OnFrameSafe;
m_objFile.PropsSet("object::on_frame.data", "true");
m_objFile.PropsSet("object::on_frame.sync", "true");

private void OnFrameSafe(string bsChannelID, object pMFrame)
{
   // ... Here should be the logic of inserting CC608 ANC data.
   System.Runtime.InteropServices.Marshal.ReleaseComObject(pMFrame); // You should call these methods to release MFrame object
   GC.Collect(); // Force CG to collect the garbage (optional) 
}

With this code, we get an access the individual frames of the MPlatform SDK's object. To get more information about events in MPlatform can be found in this article → https://support.medialooks.com/hc/en-us/articles/360000209812-Events

Now, the main part of it. To insert the custom CC608 through the ANC packet with a frame, it's needed to insert the data into the frame and put this frame into the MFRendererClass / MRendererClass or MFWriter / MWriter instance. The following code inserts empty CC608 data into ANC packets inside the frames.

MFormats & MPlatforms:

// Inserting the ANC data into the frame:
IntPtr PTR_C608;
PTR_C608 = new IntPtr(0x8080);
(pFrame as IMFFrame).MFDataSet(@"C608", 0, (Int64)PTR_C608);

The same can be done through a byte array:

// Inserting the ANC data into the frame:
byte[] dwBytes = { 0x80, 0x80 };
IntPtr pdwBytes = Marshal.AllocHGlobal(dwBytes.Length);
Marshal.Copy(dwBytes, 0, pdwBytes, dwBytes.Length);
(pFrame as IMFFrame).MFDataSet(@"C608", dwBytes.Length, (Int64)pdwBytes);

The MFDataSet() method sets both the CC608 commands and data. Although our SDKs ignore empty CC608 data inside ANC packets, we still recommend sending them for better compatibility with older equipment, especially when using CC608 but having no data to insert.

Now, let's take a look at an example of transmitting the string "Hello World!". As mentioned at the beginning of the article, the inserting process starts by sending the initialization commands. Each command consists of two bytes. Bit parity must also be applied to the command bytes! To begin the transmission, two byte sequences must be sent:

• The first command — Roll Up 3 (scroll size) — 0x26 and 0x14 — the first CC channel.

• The second command — PAC (Preamble Address Code) — 0x6011, CC1, Row 15, default style.

• The third command (optional) — CR (Carriage Return) — 0x2D14

Once these initialization commands are sent, the receiver is ready to receive the character data. Since only two characters can be sent per frame, to transmit the string "Hello World!" (which is 12 characters in total), 6 CC608 ANC packets are required. Plus 2 or 3 packets for initializing the CC608. Therefore, to transmit this string, the following CC608 data must be packed inside the ANC packets:

On the frame level, it looks the following way (all data and commands bytes must be passed with the bit parity already applied!):

Now, the only thing to do is designing the code which automatically parses our strings to be transited, into byte sequence and transmits all the necessary commands and char data to the receivers (Writers or Renderers).  These samples are included in our SDK and can be found at the following path:

MFormats SDK -> "C:\Program Files (x86)\Medialooks\MFormats SDK\Samples\C#\Sample CC608 ANC Sender\Sample CC608 ANC Sender.sln"

MPlatform SDK -> "C:\Program Files (x86)\Medialooks\MPlatform SDK\Samples Basic\C#\ANC CC608 Sender Sample\ANC CC608 Sender Sample.sln"

Using writing samples with CC plugin and the mentioned samples, you can observe how it works. Feel free to study them for a better understanding but note, that those samples are basics ones and might not be suitable for production. They only show the principles of CC608 transmitting via ANC packets.

Transmitting the string:

Besides the text, you can also send commands for CC management. For example, byte 0x21 overwrites the last char (byte 0x14 is needed for addressing to the first CC channel). So that, if you send 0x2114 (0x21 - backspace (overwrite last char), 0x14 - the first CC channel), the last char is overwritten:

In the similar way, byte 0x2D is in charge of carriage returning (scroll lines up):

The complete list of the commands and the char hex codes can be found here → https://en.wikipedia.org/wiki/EIA-608

Receive CC608 through SDI

First of all, it's needed to enable receiving ANC packet by SDI cards. To do that, follow this path in your registry "Computer\HKEY_CURRENT_USER\SOFTWARE\Medialooks\MFormats\MFLive\BMD" and set these three properties as "true":

The last folder of the path is defined the SDI card, what you use. In this article, BMD card is used. If you have, for example, BlueFish card, the path will be:

"Computer\HKEY_CURRENT_USER\SOFTWARE\Medialooks\MFormats\MFLive\BF"

For these other cards that three properties might have other names and they are true by default.

The same can be done through the code:

// For receiving side:
m_objLive.PropsSet("vanc.capture", "true");
m_objLive.PropsSet("vanc.capture_anc", "true");
m_objLive.PropsSet("vanc.capture_cc", "true");

// For receiving side:
m_objLive.PropsSet("device::vanc.capture", "true");
m_objLive.PropsSet("device::vanc.capture_anc", "true");
m_objLive.PropsSet("device::vanc.capture_cc", "true");

Also don't forget to make sure that the SDK shows CC608, not CC708, if you want to display the received closed captions:

"Computer\HKEY_CURRENT_USER\SOFTWARE\Medialooks\MCCDisplay" → cc_type = 608