Error: Macro BackLinksMenu(None) failed

compressed data is corrupt

Error: Macro TicketQuery(summary=GROUP_MEDIA_R0, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed

current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

The aim of this group is to finilize what has been started about media and to add some new things. (controls, engine behavior etc.)

Task requirements

• There should be good concept for the modularity, extensions and extension points, allowing independent media visualization and controls.
  • Create play, stop and pause buttons to manipulate media.

• Examine and improve if needed the media resources, media frame contents, etc.

• Examine and improve the existing base classes and interfaces.

• Examine the source code implementation with JMF.

• The view should create and update the videoSceneElements according to the media inserted no matter which library is used for their implementation (JMF for the time being). This includes proper behavior (no need to click the frame to update the frame of a movie for example), playing video when frame is rotated.
• See : VIDEO_CONTENT_R0
• Fix the broken unit test in *.main.media.fobs module.

Task result

• The result of this task should be source code.

Implementation idea

• Examine the existing classes for media and improve them.
• Review the scene elements for media and update them on media playing.
• Create controls for manipulation of media.

Related

SCENE_VIDEO_POINT_R0
SCENE_AUDIO_POINT_R0
AUDIO_CONTENT_R0
VIDEO_CONTENT_R0
MEDIA_ENGINE_IMPL_COMMON_R0
MEDIA_ENGINE_IMPL_MAIN_R0
MEDIA_ENGINE_IMPL_ALTERNATIVE_R0
MEDIA_FRAME_CONTROLS_R0

How to demo

• Insert a movie or an audio frame.
• Test the controls for manipulation of media.

Design

^{Describe your design here.}

Implementation

^{Describe and link the implementation results here (from the wiki or the repository).}

Testing

^{Place the testing results here.}

Comments

^{Write comments for this or later revisions here.}

Tasks of this group

MEDIA_FRAME_CONTROLS_R0

Error: Macro BackLinksMenu(None) failed

compressed data is corrupt

Error: Macro TicketQuery(summary=MEDIA_FRAME_CONTROLS_R0, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed

current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

When playing audio or video, the user needs the ability to control the media. This can be though simple controllers that start or stop the media, slider controllers that allow the user to control where in time the media is playing, or any other kind of controller we can imagine.

Each content should be able to determine how to render itself based on it's state.

Frames need a mechanism for handling adornments in general, of which controllers are one type of adornment. Adornments can either render themselves on top of the frame, or extend the size of the frame without affecting the content of the frame.

For example, a slider controller with a play/stop button, rewind, fast forward and a slider with a thumb, may live below the content in a frame. This controller would live below video. So if a video is 320x160, and the controller is 16 pixels tall, the actual frame size is 320x176. This is necessary because when the author flows text around the frame, they don't want the text flowing under a video controller.

However, other controllers may not need to extend the frame. One can imagine a controller that shows translucent controls on top of the media as the cursor passes over the media, and then fade out when the user stops moving the mouse.

So, adornments, be them controllers or some other use of adornments, need to be able to specify where they are to be rendered in the frame. They should be able to specify bottom, top, left, right, what their size should be (fixed, expanding in either vertical or horizontal dimensions) and whether or not they live inside or outside of the frame (the visual representation of it).

Adornments are properties of a frame, not the content. The same content can be rendered in 3 different frames, some with adornments, some without.

Users should be able to toggle controllers on and off.

Controllers will need to be able to interface with the content of the frame, so there will need to be a clean way to notify the controller that the content has changed state.

Task requirements

• Create controllers for media manipulation.
  • play, stop and pause.
  • "Buttons" can be active and not active - if a button is not active its SceneElement's color is more transparent than the colors of the active buttons.
  • We should have one button that is for play and pause. Its icon should be changed correspondingly to the state of the media:
    • When the media is playing the pause icon should be set to the button.
    • When the media is stopped or paused the play icon should be set.
  • The stop button should be active only when the media is playing.
  • The start button should be active only when the media is stopped or paused.
  • Clicking the play/pause button when the media is stopped will start the playback from the beginning. If the media is paused it should continue from where stopped. The same button is used to pause the media and play it.
  • Clicking the stop button when the media is started will stop the playback and set the time of media to the beginning. Clicking this button when the media is pause will do the same.
• Allow controllers to communicate with a frame's content.
• Think about how controllers can be skinned
• Notes:
  • Keep the connection between controllers and media as simple as possible.
  • Remember that every frame can have the same content in different states.

Task result

The result of this task is source code.

Implementation idea

• Review the scene elements of the media.
• Decide how to display controllers and connect them with the media manipulation interfaces (perhaps through the LogicR3).

Related

• SCENE_COMMONS_R1
• SCENE_COMMONS_R0
• MEDIA_ENGINE_IMPL_COMMON_R0
• MEDIA_ENGINE_IMPL_MAIN_R0
• VIDEO_CONTENT_R0
• AUDIO_CONTENT_R0

How to demo

• Insert a media frame.
• Click on the frame.
• Use the controls to manipulate the media.

Design

• Media frame controls will be defined as MediaManipulationViews which have image SceneElements containing icons. These will represent "buttons" that manipulate the media in Sophie2. In order for the scene elements to behave as buttons we should specify some logic that changes the alpha values of the scene elements (so that they look as if they are clicked, pressed or active - as real buttons do).

• For the time being we will implement two MediaManipulationViews:
  • PlayPauseMediaManipulationView - for playing/pausing media.
  • StopMediaManipulationView - for stopping the media.
• Every MediaManipulationView has a parent MediaContentView.
• In order for these to be skinable we should define tool-tip and icon in a SkinPart.
  • They will be defined in the two implementing classes. Icons should be put in the same module as the views.
• The MediaManipulationViews will also determine whether they should be active or not depending on media's playback.
• "Buttons" should be arranged correspondingly in the bottom left corner of the frame in the same order as described above.
• Every icon should be scaled to look nicely inside the frame.
• Media controls will be zoom-able along with the media frame but should not change their size as media frame's size changes.
  • In order to do so, make the scene element of the MediaContentView a composite one and add the media scene element as well as the scene elements of the controls to it. Every descendant of the MediaContentView (VideoContentView and AudioContentView for the time being) will provide a media element instead of just an element. Whether the media is playing or not will be kept in a value property inside the MediaContentView. This is needed for the ManipulationViews to update their state and appearance correspondingly (alpha values of colors for their scene elements).
• The changing of the alpha values of the scene elements will be done by a MediaManipulationLogic that is common both for Audio and Video.
• The managing of the media playback will be done by a single logic instead of the two that we now have. This will be named MediaLogic and will be responsible for adding media and controlling it.
• The icon of every implementation of the MediaManipulationView will be automatically changed upon skin changing. For the PlayPauseMediaManipulationView the icon will be changed upon media playback as well (play and pause icons).
• The computeActivated method will be used to determine the alpha value of the MediaManipulationView's SceneElement's color.

• Logics will be put in: /modules/org.sophie2.main.func.media/src/main/java/org/sophie2/main/func/media/logic
• MediaManipulationViews - /modules/org.sophie2.main.func.media/src/main/java/org/sophie2/main/func/media/view/controls
• The icons that we will use:
  • Play icon -
  • Pause icon -
  • Stop icon -
  • Note: The icons are just the what one will see upon media playback. Nevertheless the play/pause button will use two icon while stop button will have only one.
• Additionally the MediaManipulationViews should be registered as extensions in the MainFuncMediaModule.
• Changesets:
  • 2444
  • Demo: browser/trunk/sophie2-platform/modules/org.sophie2.author/src/test/java/org/sophie2/author/MediaDemo.java
    • resources for the demo:
      • browser/trunk/sophie2-platform/modules/org.sophie2.author/src/test/resources/distrib/demo.avi - a video file(does not have sound).

The design related code has been merged to the trunk in [2604].

Implementation

• implemented according to desing. Demo included in org.sophie2.author module because of module dependencies. Audio frame added for the demo as well: browser/trunk/sophie2-platform/modules/org.sophie2.author/src/test/resources/distrib/11k16bitpcm.wav
  • 2444
  • 2447
  • 2586
  • 2603 - a uml diagram modified.

The implementation code along with the design code has been merged to the trunk in [2604].

Testing

^{(Place the testing results here.)}

Comments

Steve doesn't feel that "including the ability to set image for not-available" and "and image for not playing" are not part of this task.

desc("related to frame contents") desc("should allow customization of how media appear (play button, slider, etc)") desc("including the ability to set image for not-available, ") desc("and image for not playing")

MEDIA_ENGINE_IMPL_MAIN_R0

Error: Macro BackLinksMenu(None) failed

compressed data is corrupt

Error: Macro TicketQuery(summary=MEDIA_ENGINE_IMPL_MAIN_R0, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed

current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

The media engine is responsible for rendering audio and video. Write a module that implements the video and audio interfaces found in org.sophie2.main.media.fobs.

Audio should play regardless of whether it is being used by a scene or not. Video does not render directly to the screen. Instead the frame should be grabbed from the Player in order for the scene to display it.

It seems as if JMF is a good place to start for implementing the engine. Start by looking at:

​http://java.sun.com/javase/technologies/desktop/media/jmf/reference/faqs/index.html#what

You will also create an extension point that wraps the class and provides a mechanism for the scene to get the audio or video handler.

Task requirements

• Write a module for media playback
• Start with JMF
• Expose an extension point for access to the handlers
• Implement the Interface found in org.sophie2.main.media.fobs
• Force the scene to repaint on playing. The grabbing of the frame should happen by equal intervals.

Task result

The result of this task is source code.

Implementation idea

• Note that JMF can only do mp3 on windows, so we need to figure out what to do about this because supporting mp3 is important for other platforms as well.

Related

MEDIA_ENGINE_IMPL_COMMON_R0

How to demo

• Insert a video frame and watch the video.
• It should play smoothly.

Design

The implementation so far has the following design:

• there is a module named - org.sophie2.base.media, which defines the model for the media and two extension points - one for Video and one for Audio

• MediaInfo is the class which holds meta data for the media
• MediaState describes the current state of the media - is it playing, what is the volume, etc.
• MediaSource defines the source for the media - is has an URL and media data which is binary *there is a module named - org.sophie2.main.media.fobs, which add two extension for the presently described extension points - one for Video and one for Audio. The engine used is JMF

We need to now the MediaState throughout media's playing (useful for buttons' appearance and slide bar) and the scene should also repaint while media is playing (for videos).

• Video is providing ImmImages that are drawn while playing. We should force manually the Scene to obtain these images.
• The MediaState should also be kept somewhere and the storing of the MediaState should also go forced manually.

To create this functionality we should use a Timer which will set the MediaState and kept the Scene in sync. There will be a single Timer that iterates over the media players and sets the media state for every one of them.

• Create a class PlayerEntry that is a descendant of BaseProObject and it will be a class representing an entry for the player registry. It should keep consistent media ID, player and MediaState. The media sate should be updated manually so that it stays consistent with the current state of the player. The player registry HashMap should be changed to a ProList containing player entries. The MediaState will be set by the Timer.

The Timer instance will be in JMFVideoEngine and it will update (force redraw of the Scene and change the MediaState of every PlayerEntry) 15 times in a second. The update will happen only if there's a runnig video.

• We should iterate over the player entries and set the state for every player.
• In the SimpleSceneVisual we should take the current state for every VideoSceneElement which is also the media id for the video it displays.

This is a link to a reference video: ​http://www.sophie2.org/trac/browser/trunk/sophie2-platform/modules/org.sophie2.base.media/src/test/resources/demo.avi

How to demo is described in the Analysis section...you should have an implementation though

Implementation

Implementation is done according to the desing. Changesets:

• 2444
• 2447
• 2586
• 2603 - a uml diagram modified.

The code was merged to the trunk in [2604] along with the MEDIA_FRAME_CONTROLS_R0 code.

Testing

Comments

Alex, ​http://java.sun.com/javase/technologies/desktop/media/jmf/reference/faqs/index.html#what might be a good place to start? - Steve

desc("making the main engine better ")

Is this task about rendering media, such as video and audio? How should it work with scenes and content objects?

Publish extension for playing video handler and audio handler

org.sophie2.main.media.fobs is the interface for the engine

audio should play even if grab frames is not called

wrap jmf class in an extension and publish it