Video filters – Medialooks Help Center

Basic info

In MPlatform it's possible to use some of the native FFMPEG video filters with MPlatform objects. For MFormats, support for FFMPEG video filter was introduced in version 2.10.0.14414 and is available exclusively in the Expert and Ultimate editions. FFMPEG video filter offer advanced video processing capabilities, allowing users to apply custom video effects and transformations within the SDK framework.

MFormats SDK

The usage of vfilters in MFormats SDK is similar to working with plugins such as CG, MCCDisplay, etc. In general, it's needed to create a MFXMediaConverter instance, set its parameters, and during the frame processing pipeline, each frame is processed by this converter. Below is an example of how to add and apply the "negate" filter in MFormats SDK:

MFReaderClass m_objReader = new MFReaderClass();
MFPreviewClass m_objPreview = new MFPreviewClass();
MFXMediaConverter m_objConverter = new MFXMediaConverter();
m_objReader.ReaderOpen(pathToVideo, "");
m_objPreview.PreviewEnable("", 1, 1);

// "av_filter_graph_desc_in" - Apply video filter before video processing (e.g. video conversion)
// "av_filter_graph_desc_out" - Apply video filter after video processing (e.g. video conversion)
String VFilter_ConvProps = @"{""av_filter_graph_desc_in"":""negate""}";
(m_objConverter as IMFProps).PropsSet(@"video::xmedia_conversion_props", VFilter_ConvProps);

M_AV_PROPS m_AV_PROPS = new M_AV_PROPS();
m_AV_PROPS.vidProps.eVideoFormat = eMVideoFormat.eMVF_Custom;

int rest = 0;
do
{
    m_objReader.SourceFrameGet(-1, out MFFrame pFrame, "");
    m_objConverter.ConverterFrameConvert("", pFrame, ref m_AV_PROPS, out var pFrameConverted, out rest, "");
    Marshal.ReleaseComObject(pFrame);
    if (pFrameConverted != null)
    {
        m_objPreview.ReceiverFramePut(pFrameConverted, -1, "");
        Marshal.ReleaseComObject(pFrameConverted);
    }   
} while (rest > 0);

Important Note! The MFXMediaConverter class is implemented exclusively for 64-bit environments. Therefore, the use of the MFXMediaConverter class and consequently, vfilters are only supported in 64-bit applications.

MPlatform SDK

To implement video filters to your MPlatform object you should use the PropsSet method.

m_objMPPlaylist.PropsSet("object::vfilter", "[name_of_the_video_filter]");

For the quick test, you can open a Playlist Sample,

"C:\Program Files (x86)\Medialooks\MPlatform SDK\Sample x64 Playlist.lnk"

add a video file, make a double click on the file name, go to the bottom of the available properties list and add new property "object::vfilter".

Now you can manage the vildeo filter of the open file with the value of the "object::vfilter" property.

Note: Video filter will increase the load of your CPU. For example, i7-4770, Playlist Sample, video file 1080 30p, playback without conversion - CPU load 14-16%. With "colorbalance=rs=.3" CPU load 22-24%.

Some vfilters as "fade", can be applied to the video clips only once. If you want to play the video clip with such a vfilter once again, it's needed to reset the vfilter via .PropsSet("object::vfilter", ""); method and set this vfilter once again. Every time when change "object::vfilter" property, you completely reset all the previous settings. So that, if you want to apply more than one vfilter to your video, it's needed to set all the necessary filters at once.

If you need to apply more than one vfilters to the object, create a combined string for "object::vfilter" property, writing all the vfilters with their parameters, separated by commas. Here is an example of applying 3 vfilter to MPlaylist:

m_objPlaylist.PropsSet("object::vfilter", "negate, avgblur=sizeX=10:sizeY=4, crop=1000:1000");

List of availible video filters

Filters writing to metadata

Some FFmpeg video filters (such as 'blackdetect') write their results into frame metadata. Since 2.8.1 SDK version you can use such filters.

Below is an example of using the 'blackdetect' filter with an MFile object.

1. Before SDK version 2.9.2.14025, vfilters did not support planar formats. Therefore, in order for filters to function correctly, planar formats had to be converted by setting the corresponding property to true:

Computer\HKEY_CURRENT_USER\Software\Medialooks\MPlatform\MFile\MFileFFM
decoder.convert_planar = true

Starting from SDK version 2.9.2.14026 and later, vfilters support planar formats, and no conversion is required.

2. Add video filter to the object as usual:

m_objMFile.PropsSet("object::vfilter", "blackdetect=d=0.05");

3. Check the result using MFStrGet method in the OnFrameSafe event:

m_objMFile.OnFrame += new IMEvents_OnFrameEventHandler(m_objMFile_OnFrame);
void m_objMFile_OnFrame(string bsChannelID, object pMFrame)
{
    (pMFrame as IMFFrame).MFStrGet("lavfi.black_start", out string val);
    Console.WriteLine(val);
    Marshal.ReleaseComObject(pMFrame);
}

More info about this filter - https://ffmpeg.org/ffmpeg-filters.html#blackdetect

Using vfilters with GPU Pipeline

Starting from Video SDK version 2.11.1.14634, it is possible to use vfilters with gpu_pipeline='true'. However, there are some important points to keep in mind:

Vfilters are processed on the CPU
When using the GPU pipeline with vfilters, each video frame is copied from the GPU to the CPU. then the vfilters are applied, and after that the frame is copied back to the GPU. This extra step can reduce performance of your solution.

10-bit video limitations
When using vfilters on a 10-bit video stream in the GPU pipeline, the 10-bit color information will be lost. This happens because each frame is copied from the GPU to the CPU, where the filter is applied, and then copied back to the GPU. The CPU can only handle 8-bit video, so the original 10-bit data is reduced to 8-bit.

To avoid losing the 10-bit quality, do not apply vfilters in the main video path. Instead, you can apply vfilters to a separate MFile. For example, if you want to use a libplacebo vfilter for a preview, create a separate MFile and apply the filter there. This keeps the original 10-bit video unchanged.

For a better understanding, see the diagram below:

In the diagram above:

The input 10-bit HDR video goes into MLive SDI.
From there, it is sent directly to MWriter and MRenderer, keeping 10-bit HDR quality.
If you want to use a vfilter for a preview, for example libplacebo, you can do it without losing 10-bit color:
1. Open the 10-bit video source in a separate MFile using mp://link.
2. Apply the libplacebo vfilter on this MFile.
3. Send the filtered MFile to MPreview.

Since the vfilter is not applied in the main 10-bit path between MLive and MWriter / MRenderer, the original 10-bit HDR video remains unchanged. Only the preview branch is reduced to 8-bit for display.

Articles in this section