Converting source video and audio formats

Conversion on a source side

IMFSource interface contains also methods with "...Converted..." names. These methods do the same actions as simple methods but also internally the input frame is converted to target format.

For example, input frame with non-standard frame size (320x240) may be converted to standard 720 50p resolution.

The difference between SourceFrameGet and SourceFrameConvertedGet method is in the _pAVPropsOut parameter. This parameter represents M_AV_PROPS structure and converts input frames according to fields value of the structure.

This structure contains audio and video properties as fields audProps and vidProps. You can manage audio and video format of your frames by settings these fields values:

• vidProps - represents M_VID_PROPS structure that describes video format,
• audProps - represents M_AUD_PROPS structure that describes audio format.

For example, to convert any input frame to 720 50p video format you should call this:

M_AV_PROPS props = new M_AV_PROPS();
props.vidProps.eVideoFormat = eMVideoFormat.eMVF_HD720_50p;

And then use this code to get a converted frame:

((IMFSource)myReader).SourceFrameConvertedGet(ref props, -1, out pFrame, "");

All the list of standard possible video formats is presented in the eMVideoFormat enumeration. And it is possible to set only this field for vidProps to set standard format. If you need a custom one you should set required fields for M_AV_PROPS structure. For example:

M_AV_PROPS props = new M_AV_PROPS();
M_VID_PROPS videoProps = new M_VID_PROPS();
videoProps.dblRate = 25.0;
videoProps.fccType = eMFCC.eMFCC_ARGB32;
videoProps.nAspectX = 4;
videoProps.nAspectY = 3;
videoProps.nWidth = 640;
videoProps.nHeight = 480;
videoProps.eInterlace = eMInterlace.eMI_Field1First;
props.vidProps = videoProps;

The similar way is used for audio format:

M_AV_PROPS props = new M_AV_PROPS();
M_AUD_PROPS audioProps = new M_AUD_PROPS();
audioProps.nChannels = 8;
audioProps.nBitsPerSample = 16;
audioProps.nSamplesPerSec = 48000;
props.audProps = audioProps;

Conversion of an MFFrame object

You can use MFConvert method to convert a single MFFrame object to the desired format. 

int rest = 0;
do
{
    MFFrame convertedFrame = null;
    pFrame.MFConvert(m_avProps, out convertedFrame, out rest, "", "");

    if (convertedFrame != null)
    {
        m_objPreview.ReceiverFramePut(convertedFrame, checkBoxRenderer.Checked ? 0 : -1, "");
        Marshal.ReleaseComObject(convertedFrame);
    }
} while (rest > 0);

As a result, you have a converted frame with modified parameters. You can use the converted frame for further processing and also you have the original frame that you can use according to your purposes.

rest value specifies the number of remaining frames needed for the current operation to produce a final output. It helps control continuous frame processing without blocking or losing synchronization.

In other words, it defines whether the frame should be converted again to maintain consistent video duration and playback speed when the frame rate is altered during conversion. Since the number of frames in the original and converted sources can differ for the same duration at different frame rates, this value helps to be ensured that the video maintains its original duration and playback speed despite frame rate changes. 

Conversion of video and audio formats with MPlatform SDK

With MPlatform SDK there is the IMFormat interface that handles operations with formats. All the source and receiver objects of MPlatfrom SDK supports this interface so you can set conversion of audio and video formats. For this, you should use FormatVideoSet or FormatAudioSet methods.

You should use eMFormatType.eMFT_Convert type to make a conversion.

Here is an example of how to set up a conversion for an MWriter object:

M_VID_PROPS vidProps; 
string formatName; 
int formatIndex; 
// required to get input video properties
(pSource as IMFormat).FormatVideoGet(eMFormatType.eMFT_Input, out vidProps, out formatIndex, out formatName); 
vidProps.eVideoFormat = eMVideoFormat.eMVF_Custom; // required for conversion 
vidProps.nWidth = 600; // set custom resolution
vidProps.nHeight = 400; 
vidProps.nRowBytes = 0; // required for conversion 
vidProps.dblRate = 15.0; 
// set the format
(m_objWriter as IMFormat).FormatVideoSet(eMFormatType.eMFT_Convert, ref vidProps); 

So you can set required values for resolution, frame rate etc.

Another example about an MPlaylist object conversion:

M_VID_PROPS props = new M_VID_PROPS(); 
props.eVideoFormat = eMVideoFormat.eMVF_Custom; 
props.dblRate = 25; 
props.nAspectX = 9; 
props.nAspectY = 16; 
props.nHeight = 1920; 
props.nWidth = 1080; 
m_objPlaylist.FormatVideoSet(eMFormatType.eMFT_Convert, ref props);

Audio conversion specifics

There are some specifics when you are using audio downmixing. For example, for 8-channels and 6-channels sources:

And similar behavior will be for 16 channels, all other channels will be mixed into first two channels. As a result you will hear louder audio there.

So, if you need to cut off other channels, consider using audio_channels property - https://support.medialooks.com/hc/en-us/articles/360000225571-Working-with-audio-through-properties 

Force conversion

Keep in mind that by default, if your source format and desired video format are equal, no conversion will be applied (even if you use SourceFrameConvertedGet/FormatVideoSet methods). To change this behavior, you can use force_conversion property available for MFile/MFReader, MLive/MFLive and MWriter/MFWriter objects. For example:

Computer\HKEY_CURRENT_USER\Software\Medialooks\MPlatform\MFile
force_conversion = true // force conversion for all files