Error: Macro BackLinksMenu(None) failed

compressed data is corrupt

Error: Macro TicketQuery(summary=BROWSER_CONTENT_BASIC_R2, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed

current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

The goal of this task is to create a native C++ wrapper for delivery of browser content.
The idea is to use WebKit, a portable browser engine to create a native browser that is able to communicate with Java.
Communication between C++ and Java is described in GROUP_WRAPPERS_R0.
The browser runs a windowless Qt application and listens for commands, once a command is received it sends a response which optionally has a payload.
For now the browser won't be able to play Flash, Flash content will be displayed in a different kind of frame.
In future revisions of this task Sophie 2 browser frame will be connected with the native wrapper and browser content will be available in Sophie 2.

Task requirements

The native browser should be able to:

• Open a web page.
• Return a captured webpage.
• Receive mouse and keyboard events.
• Resize its main window.

Task result

C++ and Java source code.
Native executables for Linux, Linux 64bit, Windows, Windows 64bit, Mac OS X, Mac OS X 64bit.

Implementation idea

Use Qt implementation of WebKit to create a browser and C++ and Java wrappers for communication.

Related

GROUP_WRAPPERS_R0

How to demo

• Start the browser demo.
• Open a webpage with forms.
• Enter some text and submit.
• Click a hyperlink.

Design

The browser consists of a Java and a native part. They communicate via the mechanism described in GROUP_WRAPPERS_R0.

Java part

• NativeBrowserBridge extends NativeBridge. This is the browser wrapper that clients should use to get and control web content.
• In future revisions of this tasks client should be able to check if the page has been successfully loaded but for now empty images can be returned.
• Internally the bridge has four commands that should be implemented - load, grab page, send mouse event and send keyboard event.
  • The grab page command returns a PageGrabResponse that contains a BufferedImage corresponding to the current web view in the native part.

---

C++ part

• The C++ part is built atop Qt, using QtWebKit which supplies a library of widgets with web functionality.
• BrowserBridge inherits abstract JavaBridge. Its role is to stand between the Java implementation and the native browser receiving commands, translating them to actions and returning the result. Browser bridge should be able to execute in its own thread as the main thread needs to run the Qt application.
• The structure of commands and responses mimics that of the Java part.
• Four commands should be implemented - load, grab page, mouse and keyboard. Two more should be added in the future - get info and resize.
  • Load - gets an url as parameter. Executed on the Browser object it causes a page to be loaded and returns without blocking. This means that the Java application should use some other way to ensure that the page is loaded, for instance use the get info command. The result from this command is either Ok or Error.
  • Grab page - executed on the Browser object. The result from this command is either PageGrabResponse or Error. PageGrabResponse contains image width, height and data.
  • Mouse - executed on the BrowserWebView. Transfers a Java mouse event to the web view. Result is Ok or Error.
  • Keyboard - executed on the BrowserWebView. Transfers a Java keyboard event to the web view. Result is Ok or Error.

---

A test has been added - NativeBrowserBridgeTest, change set: [5916].

Implementation

Protocol

OK Message

OK response ID is 0.

OKMessage

Integer offset	0
0	Response ID

Error Message

Error response ID is 1.

ErrorMessage

Integer offset	0
0	Response ID

Load Command

Load command ID is 100.

LoadCommand

Integer offset	0	1	2-
0	Command ID	URL Size	URL

Grab Page Command

Grab page command ID is 101.

GrabPageCommand

Integer offset	0
0	Command ID

Page Grab Response

Page grab response ID is 100.

PageGrabResponse

Integer offset	0	1	2	3-
0	Response ID	Width	Height	Pixel Data

Send Mouse Event Command

Send mouse event command ID is 103.

MouseIds

Name	Value
MOUSE_DBL_CLICKED	0
MOUSE_PRESSED	1
MOUSE_RELEASED	2
MOUSE_MOVED	3
MOUSE_LEFT_BUTTON	0
MOUSE_RIGHT_BUTTON	1
MOUSE_MID_BUTTON	2

SendMouseEventCommand

Integer offset	0	1	2	3	4
0	Command ID	Event	Button	X offset	Y offset

Send mouse command valid responses are Ok and Error.

Send Keyboard Event Command

Send keyboard event command ID is 104.

KeyIds

KEY_PRESSED	0
KEY_RELEASED	1

SendKeyboardEventCommand

Integer offset	0	1	2	3
0	Command ID	Event	Key	Modifiers

Send keyboard command valid responses are Ok and Error.

Resize Command

Resize command ID is 102.

ResizeCommand

Integer offset	0	1	2	3
0	Command ID	Width	Height

Resize valid responses are Ok and Error.

Send Wheel Event Command

Send wheel event command ID is 106.

SendWheelEventCommand

Integer offset	0	1	2	3	4
0	Command ID	X	Y	Delta

Send wheel event valid responses are Ok and Error.

Get Info Command

Get info command ID is 105.

GetInfoCommand

Integer offset	0
0	Command ID

Get info command valid responses are InfoResponse and Error.

Info Response

Info response ID is 101.

InfoResponse

Integer offset	0	1	2	3	4-
0	Response ID	Width	Height	URL Size	URL

Testing

^{(Place the testing results here.)}

Comments

^{(Write comments for this or later revisions here.)}