HTML5 Overlay: browser control and events

Use IMFBrowser to load HTML graphics, update a running page, forward operator input and exchange notifications with your application. The plugin's cef_plugin JavaScript bridge adds page-to-host events, host-event subscriptions and optional frame/image integration. Typical uses include data-driven lower thirds, scoreboards, interactive dashboards and application-controlled HTML graphics.

For shared SDK prerequisites, see Before you start. This reference covers browser control; the overview covers attaching/processing the plugin in a video pipeline.

This is not the MLCG REST API. If you display an MLCG output URL, use the MLCG REST Client to select sessions, change bindings and control animation. Avoid changing MLCG's internal DOM or starting its animation with injected JavaScript. The MLCG HTML REST samples demonstrate the supported separation: the SDK renders HTML, while REST controls graphics.

Choose the control path

Task / use case	API or bridge	Integration contract
Load a local/hosted graphics page or generated HTML	BrowserPageLoad, BrowserHtmlDisplay	Navigation is asynchronous. Initialize first, then wait for application-specific readiness.
Update text, style or application state without navigating	BrowserJavascriptExecute, BrowserCSSSet	Target your own page's elements/functions; serialize input and use events for acknowledgements.
Implement an interactive HTML operator panel	BrowserMouse*, BrowserKeyEvent, BrowserCommand	Map preview coordinates to video/page geometry and preserve focus/input sequences.
Send page actions/results to C# or forward host events to a page	OnEventSafe, put_event, set_on_event	Validate channels/payloads; copy notifications out of callbacks without blocking the media path.
Observe SDK frame metadata or send a diagnostic image to HTML	set_on_frame, IMCallback.OnFrame / set_on_image	Metadata updates and image transfers are different operations; neither is an automatic frame-accurate browser clock.
Prepare, navigate, recover and shut down safely	Readiness protocol, BrowserClose and owner cleanup	Page loaded, assets ready, browser painted and video presented are separate milestones.

For composition position, opacity, page size, cache/log configuration and version-specific CEF options, use the properties reference. Supporting installed sample paths and a self-contained developer test page are included after the API sections.

Load and control a page

The examples use MFORMATSLib.MFOverlayHTMLClass. With MPlatform, use the same methods from MPLATFORMLib. Keep the plugin alive and issue browser commands through the application's SDK-owner thread/queue, not from an SDK event callback.

Method	Use
BrowserPageLoad(string filenameOrUrl)	Navigate to a page. Successful return means the request was accepted, not that the page has finished loading or painting.
BrowserHtmlDisplay(string html)	Display HTML text. Prefer a properly hosted page when it has relative assets, modules or network dependencies.
BrowserJavascriptExecute(string script)	Run JavaScript in the page. There is no JavaScript return-value out parameter; use a page event for acknowledgements/results.
BrowserCSSSet(string css)	Inject CSS text into the page. It does not update the application's template model or MLCG bindings.
BrowserCommand(eMFBrowserCommand command, string parameters)	Reload, navigate back/forward, or stop a pending load.
BrowserClose()	Close browser resources during deliberate teardown. Stop processing/using the plugin first; do not close it from its own callback.

// On the SDK-owner thread, with html already created and the
// browser-controls.html test page linked below loaded and ready:
html.BrowserCSSSet("#headline { color: white; font-weight: 700; }");

// Serialize values instead of concatenating untrusted text into JavaScript.
string text = "Example headline";
string literal = new System.Web.Script.Serialization.JavaScriptSerializer()
    .Serialize(text);
html.BrowserJavascriptExecute(
    "document.getElementById('headline').textContent = " + literal + ";");

The headline element is defined in the test page below. The C# serialization example requires a reference to System.Web.Extensions in a .NET Framework application. For another page, use its own documented elements/functions; do not paste these element IDs into an arbitrary website. Expose a small, explicit page function for more complex updates and do not evaluate arbitrary remote code.

Load a document with BrowserPageLoad when it has an asset/base URL. For a small generated document with no relative assets, BrowserHtmlDisplay accepts the HTML itself:

html.BrowserHtmlDisplay(
    "<!doctype html><html><meta charset='utf-8'>" +
    "<body style='margin:0;background:transparent'>" +
    "<div id='headline' style='background:#173247;color:white;" +
    "padding:24px;font:28px Arial'>Ready</div></body></html>");

Do not immediately issue dependent script calls and assume the new document exists. Wait for the required page/application state first. Navigation commands are separate user/application actions; reloading directly after a text update would discard the change:

html.BrowserCommand(eMFBrowserCommand.eMFBC_Reload, "");
// Other commands: eMFBC_GoBack, eMFBC_GoForward, eMFBC_StopLoad.

Receive browser and page events in C#

Subscribe to OnEventSafe before loading the page. Its C# callback has four arguments: channel, event name, event parameter, and event-object context. Keep the delegate instance so that the same subscription can be removed later.

using System;
using System.Collections.Concurrent;
using MFORMATSLib; // Use MPLATFORMLib for the MPlatform edition.

// This helper observes a plugin owned by the application; it does not own it.
public sealed class BrowserNotifications
{
    private readonly MFOverlayHTMLClass _html;
    private readonly IMEventsEvent_OnEventSafeEventHandler _handler;
    private readonly ConcurrentQueue<Tuple<string, string, string>> _events =
        new ConcurrentQueue<Tuple<string, string, string>>();

    public BrowserNotifications(MFOverlayHTMLClass html)
    {
        _html = html ?? throw new ArgumentNullException(nameof(html));
        _handler = OnBrowserEvent;
        _html.OnEventSafe += _handler;
    }

    private void OnBrowserEvent(string channel, string name,
        string parameter, object eventObject)
    {
        // Copy managed values only. No SDK calls, UI invocation, network I/O,
        // blocking waits, or plugin teardown inside this callback.
        _events.Enqueue(Tuple.Create(channel, name, parameter));
        // Do not retain or final-release the shared event-object context.
    }

    public bool TryRead(out Tuple<string, string, string> notification)
    {
        return _events.TryDequeue(out notification);
    }

    // Call from the owner after stopping processing and draining callbacks.
    public void Detach()
    {
        _html.OnEventSafe -= _handler;
    }
}

Drain the queue from your normal application loop/dispatcher and update controls there. For a high-rate custom event, filter or coalesce notifications rather than allowing an unbounded backlog. Treat event data as input: validate the channel, event name and payload before performing any action. The event object can be the same COM object already owned by the application; it is not a new plugin to close or final-release.

Channel / event	Meaning
CEF / on_load_start	A frame began loading. The parameter is that frame's URL.
CEF / on_load_end	A frame's load ended. The parameter is its URL. Subframes can also generate this event; it is not a “graphics ready” or fresh-GPU-frame acknowledgement.
CEF / on_load_error	A frame load failed or was aborted. The parameter is the failed URL; use logs for further diagnosis. Do not treat every aborted navigation as an application failure.
Application-defined channel and event	A notification emitted by the page with cef_plugin.put_event.

Send a notification from JavaScript

cef_plugin is injected by the SDK's browser runtime; it is not present in an ordinary desktop browser. The bridge carries strings:

if (window.cef_plugin) {
    cef_plugin.put_event("my-overlay", "score-changed",
        JSON.stringify({ home: 2, away: 1 }));
}

The application receives channel my-overlay, name score-changed, and the JSON text as the event parameter. JSON here is an application-chosen payload, not an MLCG REST request. Do not put credentials in these notifications or log sensitive page data indiscriminately.

Readiness and resource preparation

A page can finish navigation while its application data, fonts or images are still being prepared. For your own page, send a bounded, application-specific readiness signal after those dependencies are usable. For example, after your data has been applied:

async function notifyOverlayReady(instanceId) {
    try {
        if (document.fonts && document.fonts.ready) {
            await document.fonts.ready;
        }
        await Promise.all(Array.from(document.images).map(function (image) {
            if (image.decode) return image.decode();
            if (image.complete) {
                return image.naturalWidth > 0 ? Promise.resolve() :
                    Promise.reject(new Error("Image is unavailable"));
            }
            return new Promise(function (resolve, reject) {
                image.addEventListener("load", resolve, { once: true });
                image.addEventListener("error", reject, { once: true });
            });
        }));
        requestAnimationFrame(function () {
            if (window.cef_plugin) {
                cef_plugin.put_event("my-overlay", "ready", instanceId);
            }
        });
    } catch (error) {
        if (window.cef_plugin) {
            cef_plugin.put_event("my-overlay", "not-ready", instanceId);
        }
    }
}

Use a new instance/request identifier for each navigation and accept only the matching response. Apply an application-side timeout and cancellation; readiness must not block a callback or stop the UI indefinitely. Define how the page treats optional images, failed required fonts and dynamically added assets. Font readiness and image decoding are separate browser operations.

This signal means application resources are prepared; it does not prove that the corresponding pixels have reached the SDK output. requestAnimationFrame is not a GPU-paint fence. If the workflow requires frame-accurate synchronization, verify the actual processed/presented frames and timestamps. For MLCG output, use its supported readiness/playback workflow rather than copying this custom-page protocol or restarting playback on every load event.

Handle SDK frame metadata in JavaScript

For a page that intentionally follows SDK frame processing, register a global function name as a string. The callback receives a serialized SDK parameter string, not a JavaScript object or JSON document. Parse it with cef_plugin.parse_parameters:

Put this code in the page's script before DOMContentLoaded, and add a timecode display element if required. Adding a new DOMContentLoaded listener after a page has already loaded does not rerun that event.

function on_frame_my(frameInfo) {
    if (!frameInfo || !window.cef_plugin) return;
    var info = cef_plugin.parse_parameters(frameInfo);
    var timecode = document.getElementById("timecode");
    if (timecode) timecode.textContent = info.timecode || "";
    // info.time and info.vid_props are formatted SDK strings as well.
}

document.addEventListener("DOMContentLoaded", function () {
    if (window.cef_plugin) cef_plugin.set_on_frame("on_frame_my");
});

Use a classic/global script for the named callback, or explicitly attach the function to window. Keep it short: avoid network requests, synchronous property reads, large DOM rebuilds, and unnecessary layout measurements. A browser callback on each update does not by itself guarantee one newly painted page frame per video frame. Interlaced processing can also have field-related updates; do not advance an animation by assuming a fixed callback count.

Handling the OnEvent events

set_on_event subscribes a page function to events forwarded by the plugin/host. The payloads are strings; optional parameters can be empty or absent depending on the event and SDK build. The object description is not a COM object that JavaScript can call.

function on_event_my(channel, name, parameter, objectDescription) {
    var label = document.getElementById("last-event");
    if (label) label.textContent = channel + ": " + name;
    // Validate names and payloads before acting on application events.
}

if (window.cef_plugin) cef_plugin.set_on_event("on_event_my");

Host-specific information exists only when the overlay is attached to a compatible host. For example, an MPlatform playlist can expose paths such as host::current::file::info::path and host::next::file::info::path; an MFormats frame processor does not become an MPlatform playlist.

MPlatform can forward relevant host events to the plugin. For a deliberately application-defined notification, call the plugin's IMCallback interface on the SDK-owner thread; the page receives it through its registered set_on_event function:

((IMCallback)html).OnEvent(
    0, "my-overlay", "refresh-requested", "current-data", null);

cef_plugin.props_get(name) performs a synchronous property request in this bridge. Use it sparingly for event-driven diagnostics; never put it in a per-frame animation loop. props_set(name, value) sends a property write and is not a REST call. Restrict both APIs to trusted page code and known property names.

Handling the OnImage events

When the SDK sends an image to a subscribed page, the callback receives an image name and an ImageData object. This is an image-data transfer to JavaScript, not zero-copy GPU texture access. Register the callback only when the page needs these images:

function on_image_my(imageName, imageData) {
    var canvas = document.getElementById("received-image");
    if (!canvas || !imageData) return;
    canvas.width = imageData.width;
    canvas.height = imageData.height;
    canvas.getContext("2d").putImageData(imageData, 0, 0);
}

if (window.cef_plugin) cef_plugin.set_on_image("on_image_my");

The example displays at native image size. Scale using a separate canvas if required, and avoid allocating large temporary canvases on every frame. Merely subscribing does not make arbitrary host video frames arrive; the application/host must send the intended image.

To supply an image from an SDK frame your application already owns, use IMCallback.OnFrame after the page has registered set_on_image. This named image-delivery operation is distinct from set_on_frame's timing/format notifications:

public static class BrowserImageBridge
{
    public static void Send(MFOverlayHTMLClass html, object sourceFrame)
    {
        if (html == null) throw new ArgumentNullException(nameof(html));
        if (sourceFrame == null)
            throw new ArgumentNullException(nameof(sourceFrame));
        ((IMCallback)html).OnFrame(0, "diagnostic", sourceFrame);
    }
}

sourceFrame must be a valid SDK frame, not a bitmap or URL; use the matching SDK family and your application's normal frame ownership. The helper borrows the supplied frame and does not release it. Image delivery is asynchronous and can be busy; it is not an acknowledged per-frame transport queue. Use it for intentional page image/diagnostic features, not as a zero-copy replacement for the SDK video-composition pipeline. Frequent full-frame transfers can add memory copies and JavaScript work.

Mouse, keyboard, and navigation

The page is rendered off-screen, so interacting with an SDK preview is not equivalent to clicking a normal browser window. Forward input through BrowserMouseClick, BrowserMouseMove, BrowserMouseWheel and BrowserKeyEvent when your application needs interactive HTML.

C# method	Input contract
BrowserMouseClick(ref MF_MOUSE_EVENT mouse, eMFMouseButtons button, int mouseUp, int clickCount)	Use 0 for button-down and 1 for button-up; forward both. The example below sends one left click.
BrowserMouseMove(ref MF_MOUSE_EVENT mouse, int mouseLeave)	Forward the mapped position and indicate when the pointer leaves the page area.
BrowserMouseWheel(ref MF_MOUSE_EVENT mouse, int deltaX, int deltaY)	Forward the mapped pointer location and wheel deltas, such as the UI mouse-wheel event's delta.
BrowserKeyEvent(ref MF_KEY_EVENT key)	Supply the correct event type, key/character fields and actual modifier state. Character input needs an editable page element to have focus.

MF_MOUSE_EVENT.bAbsolutePos = 0 selects normalized coordinates from 0.0 to 1.0; on the overlay object these are interpreted against the video frame and adjusted for its destination rectangle. Convert preview coordinates to video coordinates first, excluding letterboxing. Absolute coordinates and crop/zoom need explicit mapping to the browser viewport. Forward both mouse-down and mouse-up, and preserve keyboard modifiers and character events.

Do not apply a global dx11.windowless registry change as a universal mouse fix. Input routing depends on the preview renderer and SDK build; start with explicit input forwarding and test it on the actual output geometry. See getting started for pipeline setup and properties reference for destination coordinates.

For a full-frame overlay, this example sends a left click at the center of the video. Run it on the SDK-owner thread after the page is ready; no preview-window mouse hook or registry change is required to send the input explicitly.

var mouse = new MF_MOUSE_EVENT
{
    bAbsolutePos = 0, // Normalized video coordinates, not desktop pixels.
    dblPosX = 0.5,
    dblPosY = 0.5,
    eFlags = eMFEventsFlags.eMFEF_LEFT_MOUSE_BUTTON
};
html.BrowserMouseClick(ref mouse, eMFMouseButtons.eMFMB_Left, 0, 1); // Down.
mouse.eFlags = 0;
html.BrowserMouseClick(ref mouse, eMFMouseButtons.eMFMB_Left, 1, 1); // Up.

Change the coordinates to target a specific page control. Keyboard character input additionally requires an editable page element to have focus; sending a character is not a substitute for focusing the field or for forwarding a complete physical key sequence.

For example, after focusing an editable page field, this sends a single A character. It does not synthesize modifier shortcuts or the complete key-down/key-up sequence:

var key = new MF_KEY_EVENT
{
    eType = eMFKeyEventType.eMFKT_Char,
    windows_key_code = 'A',
    character = 'A',
    unmodified_character = 'A'
};
html.BrowserKeyEvent(ref key);

Threading and cleanup checklist

1. Keep SDK browser calls on the application's SDK-owner path; keep event callbacks short and non-blocking.
2. Stop accepting new page/UI commands before shutdown. Stop processing or detach the plugin from its host and drain in-flight work.
3. Unsubscribe the retained event delegate before closing and releasing the application-owned plugin.
4. Do not retain callback COM context or pass frame/COM objects to UI work without an explicit lifetime owner. Copy managed strings/statistics instead.
5. Close browser/plugin resources and release each application-owned COM object deterministically. Do not use forced garbage collection as shutdown logic.

Supporting tools: installed browser-control samples

Use these applications to inspect browser behavior while developing. These are default installed paths; use your chosen install folder if it differs. Availability and labels depend on the SDK release.

MPlatform: advanced Playlist Sample

Run:
C:\Program Files (x86)\Medialooks\MPlatform SDK\Samples Advanced\C#\Playlist Sample\bin\x64\Debug\Playlist Sample.exe

Solution:
C:\Program Files (x86)\Medialooks\MPlatform SDK\Samples Advanced\C#\Playlist Sample\Playlist Sample.sln

Browser-controls source:
C:\Program Files (x86)\Medialooks\MPlatform SDK\Samples Advanced\C#\MControls Library\OverlayHTMLWindow.cs

Add and play a video, enable HTML overlay, then open Props. This is the advanced Playlist Sample, not the basic Internal Playlist Sample.

MFormats: Sample File Playback

Run:
C:\Program Files (x86)\Medialooks\MFormats SDK\Samples\C#\Sample File Playback\bin\x64\Debug\Sample File Playback.exe

Solution:
C:\Program Files (x86)\Medialooks\MFormats SDK\Samples\C#\Sample File Playback\Sample File Playback.sln

Browser-controls source:
C:\Program Files (x86)\Medialooks\MFormats SDK\Samples\C#\Sample File Playback\OverlayHTMLWindow.cs

Use Load File to choose a video and start playback, enable HTML overlay, then open Props. Both editions provide an OverlayHTMLWindow with page address, input preview, navigation and JavaScript controls.

Self-contained developer test page

Save this as C:\Graphics\browser-controls.html for an asset-free DOM and bridge test. It defines the element IDs used in this article; those IDs must not be assumed to exist on an unrelated website or MLCG output page. Clicking Apply text also sends an applied event when running in the SDK browser.

<!doctype html>
<html lang="en">
<meta charset="utf-8">
<style>
  html, body { margin: 0; background: transparent; font: 28px Arial; }
  .card { position: absolute; left: 6%; top: 10%; padding: 24px;
          background: #173247; color: white; }
  input, button { font: inherit; margin-top: 12px; }
</style>
<div class="card">
  <div id="headline">Ready to update</div>
  <input id="message" aria-label="Headline" value="Hello from the SDK">
  <button id="apply">Apply text</button>
</div>
<script>
  document.getElementById("apply").onclick = function () {
    var text = document.getElementById("message").value;
    document.getElementById("headline").textContent = text;
    if (window.cef_plugin) {
      cef_plugin.put_event("my-overlay", "applied", text);
    }
  };
</script>
</html>

Check the page in a sample

1. In Props, enter C:\Graphics\browser-controls.html in the page address and click InitSource (MPlatform) or Init Source (MFormats).
2. Once the page appears, click its input and type, then click Apply text. The Props preview forwards mouse, wheel and supported keyboard events; focus the editable field before typing.
3. In the JavaScript box, run the following with Run Java script (MPlatform) or Run Java Script (MFormats). The card should display the new text.
4. R reloads the page; <- and -> navigate available browser history. These actions affect this browser instance, not REST data or a saved MLCG template.

document.getElementById("message").value = "Updated through JavaScript";
document.getElementById("apply").click();

Related: Getting started · Properties reference · MLCG HTML REST samples.