Basic usage of HTML5 plugin

The HTML5 Overlay plugin works just like any other plugin for MPlatform and MFormats SDKs.

Create the object

Create a new object and add it to your processing logic.

MPlatform:

// Create a plugin object
MFOverlayHTMLClass m_objOverlayHTML = new MFOverlayHTMLClass();
// Add it to the master object   
m_objPlaylist.PluginsAdd(m_objOverlayHTML, 0);

MFormats:

// Creare plugin object  
MFOverlayHTMLClass m_objOverlayHTML = new MFOverlayHTMLClass();  
// ... some threading stuff here to start receiving frames from the source ...              
 // Process each frame with plugin to apply changes   
m_objOverlayHTML.ProcessFrame(pFrame, out pFrameOut, out nFramesRes, ""); 

Load the web page

Use local web pages or web URLs:

m_objOverlayHTML.BrowserPageLoad("google.com");

Set the page display parameters: position, size, zoom, etc.

Use the HTML Plugin to configure the page appearance and behavior. For example, you can display just a part of the page and scroll down the content. The display parameters are configured using IMProps interface:

// Set the display area to the right side of the stream 
m_objOverlayHTML.PropsSet("dst.x", "60%"); 
m_objOverlayHTML.PropsSet("dst.y", "5%"); 
m_objOverlayHTML.PropsSet("dst.w", "40%"); 
m_objOverlayHTML.PropsSet("dst.h", "90%"); 

// Crop the page from the left and right sides to make it look nicely and focus on a center column of the page 
m_objOverlayHTML.PropsSet("crop", "50,0,50,0"); 

// Scroll down the page vertically to display the content outside the display area 
m_objOverlayHTML.PropsSet("scroll.speed_v", "1"); 

Control the page with mouse and keyboard actions

Use the page control methods to operate the page just like you would in a browser. As the plugin is displayed on a preview panel, we can monitor the mouse moves and clicks on that panel and pass them to the plugin to use for page control – or just simulate it programmatically.

For example, to simulate a left mouse button click:

// Let's pass the left mouse down event to the HTML plugin 
// Create mouse event structure to fill the event details 
MF_MOUSE_EVENT mouseEvent = new MF_MOUSE_EVENT(); 

// Specify the mouse position 
mouseEvent.dblPosX = 100; 
mouseEvent.dblPosY = 200; 
             
// Use the eFlags value to specify the button that is pressed now. It can be a shift, alt, etc. 
// pass the left mouse button press click. 
mouseEvent.eFlags = eMFEventsFlags.eMFEF_LEFT_MOUSE_BUTTON; 

// Pass the mouse click to the plugin  
// 3-rd parameter here (0) means that the button was pressed down (in opposite to mouse up) 
// 4-th parameter here (1) means that the clicks count. We pass 1 for a single click  
m_objOverlayHTML.BrowserMouseClick(mouseEvent, eMFMouseButtons.eMFMB_Left, 0, 1);

You can also simulate keyboard buttons:

// Let's simulate the key press events
// For example, the "A" key press
MF_KEY_EVENT keyEvent = new MF_KEY_EVENT();

// We will use the Char event type to simulate a single keystroke
keyEvent.eType = eMFKeyEventType.eMFKT_Char;
keyEvent.windows_key_code = (int)Keys.A;
m_objOverlayHTML.BrowserKeyEvent(keyEvent)

// We can also pass a key down event
// For example the "Alt" key
keyEvent = new MF_KEY_EVENT();

// Here a KeyDown event type is used to simulate a button press down
keyEvent.eType = eMFKeyEventType.eMFKT_KeyDown;
keyEvent.windows_key_code = (int)Keys.Alt;
m_objOverlayHTML.BrowserKeyEvent(keyEvent); 

Use common browser commands: back, forward, reload

Just like in any browser, you can go forward, backwards or reload the page:

// Navigate backward  
m_objOverlayHTML.BrowserCommand(eMFBrowserCommand.eMFBC_GoBack, ""); 
// Navigate forward  
m_objOverlayHTML.BrowserCommand(eMFBrowserCommand.eMFBC_GoForward, ""); 
// Reload the page 
m_objOverlayHTML.BrowserCommand(eMFBrowserCommand.eMFBC_Reload, "");

Interact with the page via JavaScript code

Use JavaScript to interact with a web page, such as fill in an email and password and click the login button:

string myJavascript = "document.getElementById('email').value = 'YourEmail';"; 
myJavascript += "document.getElementById('pass').value = 'YourPass';"; 
myJavascript += "document.getElementById('loginbutton').click();"; 
m_objOverlayHTML.BrowserJavascriptExecute(myJavascript);