Error: Macro BackLinksMenu(None) failed

compressed data is corrupt

Error: Macro TicketQuery(summary=BROWSER_CONTENT_BASIC_R2, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed

current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

The goal of this task is to create a native C++ wrapper for delivery of browser content.
The idea is to use WebKit, a portable browser engine to create a native browser that is able to communicate with Java.
Communication between C++ and Java is described in GROUP_WRAPPERS_R0.
The browser runs a windowless Qt application and listens for commands, once a command is received it sends a response which optionally has a payload.
For now the browser won't be able to play Flash, Flash content will be displayed in a different kind of frame.
In future revisions of this task Sophie 2 browser frame will be connected with the native wrapper and browser content will be available in Sophie 2.

Task requirements

The native browser should be able to:

• Open a web page.
• Return a captured webpage.
• Receive mouse and keyboard events.
• Resize its main window.

Task result

C++ and Java source code.
Native executables for Linux, Linux 64bit, Windows, Windows 64bit, Mac OS X, Mac OS X 64bit.

Implementation idea

Use Qt implementation of WebKit to create a browser and C++ and Java wrappers for communication.

Related

GROUP_WRAPPERS_R0

How to demo

• Start the browser demo.
• Open a webpage with forms.
• Enter some text and submit.
• Click a hyperlink.

Design

The browser consists of a Java and a native part. They communicate via the mechanism described in GROUP_WRAPPERS_R0.

Java part

• NativeBrowserBridge extends NativeBridge. This is the browser wrapper that clients should use to get and control web content.
• In future revisions of this tasks client should be able to check if the page has been successfully loaded but for now empty images can be returned.
• Internally the bridge has four commands that should be implemented - load, grab page, send mouse event and send keyboard event.
  • The grab page command returns a PageGrabResponse that contains a BufferedImage corresponding to the current web view in the native part.

---

C++ part

• The C++ part is built atop Qt, using QtWebKit which supplies a library of widgets with web functionality.
• BrowserBridge inherits abstract JavaBridge. Its role is to stand between the Java implementation and the native browser receiving commands, translating them to actions and returning the result. Browser bridge should be able to execute in its own thread as the main thread needs to run the Qt application.
• The structure of commands and responses mimics that of the Java part.
• Four commands should be implemented - load, grab page, mouse and keyboard. Two more should be added in the future - get info and resize.
  • Load - gets an url as parameter. Executed on the Browser object it causes a page to be loaded and returns without blocking. This means that the Java application should use some other way to ensure that the page is loaded, for instance use the get info command. The result from this command is either Ok or Error.
  • Grab page - executed on the Browser object. The result from this command is either PageGrabResponse or Error. PageGrabResponse contains image width, height and data.
  • Mouse - executed on the BrowserWebView. Transfers a Java mouse event to the web view. Result is Ok or Error.
  • Keyboard - executed on the BrowserWebView. Transfers a Java keyboard event to the web view. Result is Ok or Error.

---

A test has been added - NativeBrowserBridgeTest, change set: [5916].

Implementation

Protocol

OK Message

OK response ID is 0.

OKMessage

Integer offset	0	1
0	Size	Response ID

OKMessage Example

Integer offset	0	1
0	2	0

Error Message

Error response ID is 1.

ErrorMessage

Integer offset	0	1
0	Size	Response ID

Error Message Example

Integer offset	0	1
0	2	1

Load Command

Load command ID is 100.

LoadCommand

Integer offset	0	1	2-
0	Size	Command ID	URL

LoadCommand Example

Integer offset	0	1	2-
0	18	100	​http://google.com

Load command responses could be OK or Error.

Grab Page Command

Grab page command ID is 101.

GrabPageCommand

Integer offset	0	1
0	Size	Command ID

GrabPageCommand Example

Integer offset	0	1
0	18	101

Grab page command responses could be PageGrabResponse or Error.

Page Grab Response

Page grab response ID is 100.

PageGrabResponse

Integer offset	0	1	2	3	4-
0	Size	Command ID	Width	Height	Pixel Data

PageGrabResponse Example

Integer offset	0	1	2	3	4-
0	480004	100	800	600	data

Send Mouse Event Command

Send mouse event command ID is 103.

MouseIds

Name	Value
MOUSE_DBL_CLICKED	0
MOUSE_PRESSED	1
MOUSE_RELEASED	2
MOUSE_MOVED	3
MOUSE_LEFT_BUTTON	0
MOUSE_RIGHT_BUTTON	1
MOUSE_MID_BUTTON	2

SendMouseEventCommand

Integer offset	0	1	2	3	4	5
0	Size	Command ID	Event	Button	X offset	Y offset

SendMouseEventCommand Example

Integer offset	0	1	2	3	4	5
0	6	103	1	1	100	150

Testing

^{(Place the testing results here.)}

Comments

^{(Write comments for this or later revisions here.)}