Transparency processing

To work with transparency (alpha channel), you must convert your original object to ARGB format with:

M_VID_PROPS avprops = new M_VID_PROPS();
avprops.fccType = eMFCC.eMFCC_ARGB32;
m_objFile.FormatVideoSet(eMFormatType.eMFT_Input, avprops);

Overlay alpha

If an alpha channel is used in an image, two common representations are available: straight (unassociated) alpha and premultiplied (associated) alpha. With straight alpha, the RGB components represent the colour of the object or pixel, disregarding its opacity and alpha component represents pixel opacity. With premultiplied alpha, the RGB components represent the emission of the object or pixel, and the alpha represents the occlusion.

In this article, we will use the MFOverlayRect method to overlay frames. You can read more information about using this method here.

By default, if none of the options is selected, the alpha blending mode will be 'over_premul'. In this case, both frames must be with premultiplied alpha.

Straight on premultiplied

To overlay the frame with straight alpha to frame with premultiplied alpha, you must use the over_premul1 option. As an example, create a completely transparent frame on which we will overlay the frames for the appearance of our logo: 

// Fill RGBA 0xFFFF 00 00
m_objFactory.MFFrameCreateFromMem(ref m_AV_PROPS, 0, 0, 0, out pFrame, "solid_color='yellow(0)'");
string propsList = "alpha_blending_mode=over_premul1";
pFrame.MFOverlayRect(frameToOverlay, null, ref overlayRect, overlayFlags, overlayTransparency , propsList, "");

When we using the over_premul1 option, we get the expected and correct result:

But if we use over_premul with the same frames, as a result, we get a blending of yellow and semi-transparent pixels.

Premultiplied on straight

To overlay the frame with premultiplied alpha to frame with straight alpha, you must use the over_premul2 option.

MF_RECT overlayRect = new MF_RECT(); 
overlayRect.dblHeight = 120; 
overlayRect.dblWidth = 160; 
overlayRect.dblPosX = 30; 
overlayRect.dblPosY = 20; 
overlayRect.eRectType = eMFRectType.eMFRT_Absolute; 
double overlayTransparency = 0.9;
string propsList = "alpha_blending_mode=over_premul2"; 
eMFOverlayFlags overlayFlags = eMFOverlayFlags.eMFOF_Crop; 
pFrame.MFOverlayRect(frameToOverlay, null, ref overlayRect, overlayFlags, overlayTransparency , propsList, "");

Encoding alpha

To encoding video with alpha channel, you can use these formats and codecs:

• VP8
• VP9
• QT RLE
• FFV1
• PNG

Please note, that these codecs will require more CPU resources.

Alpha in H264 and H265

HEVC does not support the alpha channel.
But there is an option to send and receive an expanded frame (RGB and a mask in one frame). But then they will need to be stitched together during playback. Therefore, this approach is limited to your applications.

The masked approach returns an "increased" frame - the height of the frame is doubled.

private MFFrame[] setARGBmask(MFFrame frame)
{
M_AV_PROPS frame_props;
int samples;
string sideDataARGB = "_hdyc_w_mask_";

object obj;

MFFrame refCPU;
frame.MFClone(out refCPU, eMFrameClone.eMFC_Reference_ForceCPU, eMFCC.eMFCC_ARGB32);
(refCPU as IMFSideData).MFObjGet(sideDataARGB, out obj, 0);
frame = (MFFrame)obj;
releaseComObj(refCPU);

return splitAlpha(frame);
}

Cropping a part of the frame with alpha

private MFFrame[] splitAlpha(MFFrame frame)
{
    if (frame != null)
    {
        int height;
        int width;
        int audioSamples;
        M_AV_PROPS props;
        MFFrame[] frames = new MFFrame[2];
        frame.MFAVPropsGet(out props, out audioSamples);
        height = props.vidProps.nHeight;
        width = props.vidProps.nWidth;

        tagRECT top = new tagRECT();
        top.left = 0;
        top.right = width;
        top.bottom = height / 2;
        top.top = 0;

        tagRECT bottom = new tagRECT();
        bottom.left = 0;
        bottom.right = width;
        bottom.bottom = height;
        bottom.top = height / 2; ;
        frame.MFCut(0, ref top, out frames[0]);
        frame.MFCut(0, ref bottom, out frames[1]);

        return frames;
    }
    else {
        return null;
    }
}

As an example, you can send these frames to different previews.

if (pFrame != null) 
{
   MFFrame[] frames = setARGBmask(pFrame);
   if (frames[0] != null && frames[1] != null)
   {
      m_objPreview.ReceiverFramePut(frames[0], -1, "");
      m_objPreview2.ReceiverFramePut(frames[1], -1, "");
}
else
{
  m_objPreview.ReceiverFramePut(pFrame, -1, "");
}
if (m_objWriter!= null)
  m_objWriter.ReceiverFramePut(pFrame, 0, "");
 releaseComObj(pFrame);
}