Fill and Key input

Not all renderer devices support the transmission of video with an alpha channel. So that, if it's needed to receive the video with the alpha channel, two renderers can transmit two separate signals simultaneously: Fill (the RGB frame content) and Key (the alpha channel of the frame), and then combine them back into a video with an alpha channel.

So that, it's possible to overlay one live video source with another, which is used as a mask. As a result, you have a mix of 2 live sources. In essence, this is a form of reverse keying, where Fill and Key inputs are merged to produce a video with alpha.

To ensure proper blending, both sources must share the same video format — specifically, the same resolution and frame rate.

In general, mixing two sources can be done using the MMixer object in MPlatform SDK or the MFOverlay() and MFObjSet("mask", pFrameKey) methods in MFormats SDK. However, for live sources in MPlatform and any sources in MFormats, there's a dedicated feature called the "mask" type, which simplifies this process:

MPlatform SDK

The "mask" feature is only available for the MLive object. You can specify the required input source using the DeviceSet method:

m_objLive.DeviceSet("mask", deviceName, ""); // you should use a name of a device as deviceName, like "UltraStudio Express"

To specify an input line, it's needed to set "mask::line-in" device type:

m_objLive.DeviceSet("mask::line-in", lineName, ""); // you should use a name of a device as lineName, like "HDMI Video & HDMI Audio"

MFormats SDK

Because MFormats is based on frames, it is possible to use any source as a "mask" for mixing. The idea is to grab a frame from the main source, grab a frame from a mask source and then attach the "mask" frame to the main frame:

MFFrame pFrameFill;
m_objLive_Fill.SourceFrameGet(-1, out pFrameFill, "");

MFFrame pFrameKey;
m_objLive_Key.SourceFrameGet(-1, out pFrameKey, "");

// attach the mask frame to the main one
pFrameFill.MFObjSet("mask", pFrameKey);

// process the pFrameFill further
m_objPreview.ReceiverFramePut(pFrameFill, -1, "");

Important Note: When you use a Key frame in the ARGB32 format as a mask for a Fill frame, the alpha channel of the Key frame is used directly to define the transparency of the Fill frame. In contrast, for formats that do not support an alpha channel (e.g., HDYC), the luminance (brightness) of the Key frame pixels is used to calculate the alpha channel of the resulting Fill frame.

Mask frame doesn't contain alpha channel (For example: vidProps.fccType = eMFCC.eMFCC_HDYC):

Mask frame contains alpha channel (For example: vidProps.fccType = eMFCC.eMFCC_ARGB32):

Synchronizing Fill and Key SDI Sources at the Application Level

To synchronize Fill and Key signals from two separate SDI live sources (MFLiveClass or MLiveClass instances), the frames from both sources should be compared based on their hardware timestamps → hwtm_abs. This timestamp is provided by the Blackmagic Design (BMD) hardware and represents the absolute hardware timestamp for each frame, expressed in Int64 format, where each unit corresponds to 100 nanoseconds. The hwtm_abs values for each frame can be retrieved using .MFDataGet("hwtm_abs", out _, out Int64 HWTime) method.

(pFrameFill as IMFFrame).MFDataGet("hwtm_abs", out _, out Int64 HWTime_FrameFill);
(pFrameKey  as IMFFrame).MFDataGet("hwtm_abs", out _, out Int64 HWTime_FrameKey);

Note: This synchronization method works exclusively for video processed by BMD devices.

In the application, this synchronization is achieved by accumulating the frames from both the Fill and Key sources into separate buffers. These buffers store frames along with their corresponding hwtm_abs timestamps. The next step is to compare the hwtm_abs frames' timestamps from both buffers and select the pair with the closest hwtm_abs values. This approach ensures that the frames are as synchronized as possible based on their hardware timestamp differences.

Important Note: The hwtm_abs value is assigned to frames on the MLives or MFLives side at the moment the frame is received, not on the renderer side when the frame is sent. Therefore, Fill and Key synchronization using this method is only possible if the Fill and Key signals are initially synchronized — for example, when they are generated by the same BMD board operating in a corresponding mode. How to select the desired mode is covered in this article.

This approach allows for high-precision synchronization, by comparing frames with closely aligned timestamp values. To do this, an extra frame buffers at the application level is required for obtaining the frames from different MLives or MFLives with similar hwtm_abs values.