Analysis
Overview
The goal of this task is to provide basic prototype implementations for Scene performance optimizations in two directions - algorithmic and hardware.
Task requirements
Algorithmic part: This part introduces new concepts in the Scenes abstraction and implementation like sprites, tiles, etc. Some smart algorithms will be used in order to make the future implementation faster.
The new changes will include:
- Introduction of sprites and tiles,
- Integration of sprites and tiles with Scenes, SceneHelpers, SceneElements, etc.
- Introduction of a global cache for tiles,
- Introduction of some concrete effects like clipping and coloring, etc.
- and possibly others.
The visible result for the algorithmic optimizations will be a demo which:
- creates a scene,
- adds some scene elements to it,
- possibly modifies them,
- the scene gets updated at each modification.
This demo will be useful because it will show that the scene gets redrawn correctly with each modification of its model with the new concepts of sprites and tiles.
Hardware part: In this part you should research and analyze different hardware-oriented graphic libraries for improving Scenes performance.
You should give proposals for solutions for the following problems:
- How to integrate a GL lib with the Scenes hierarchy?
- How to utilize OpenGL if possible?
- Could the video memory of the graphics card be utilized?
Try researching if any of the following GLs could be used:
- JOGL
- Java3D
- JavaFX
Choose a GL from the above and explain why you've chosen it. Using a GL can be OS dependent. Propose how to solve such native issues with the chosen GL.
Provide some sort of demo like the one in the algorithmic part but with focus on altered drawing implementations which uses a GL.
Task result
- Prototype implementations of algorithmic and hardware optimizations for Sophie which can be integrated with our Scenes concept.
- Good design which will allow completion of both implementation in the future.
Implementation idea
- Use sprites and tiles concepts from Milo.
- Most likely use Java3D and read a tutorial book.
Related
SCENE_COMMONS_R0 SCENE_COMMONS_R1
How to demo
- Run both demos.
Design
Algorithmic part:
- Sprite is an immutable image in the 2d plane. If the 2d plane is separated into rectangles with a fixed size (for example 64x64) and we call them tiles, each sprite has a method to get the tile at a specified position.
- Create a new package - org.sophie2.base.scene.sprites, an abstract base class Sprite and a few extending classes (ColorSprite, ClipSprite, BlendSprite, ImageSprite, etc).
- Generally, every sprite has a source sprite and additional information about the transformation it applies on the source sprite - ImmColor if it is a ColorSprite, clipping ImmArea if it is a ClipSprite and so on.
- ElementHelpers will no longer have a drawContent(Graphics2d g) methods. Instead, they will generate sprites. Three new methods will be added in ElementHelper:
- abstract Sprite getOwnSprite() - generates a sprite that can show only this element (without its children or effects).
- Sprite getCompositeSprite() - generates a sprite that can show this element and all its children.
- Sprite getFinalSprite() - generates a sprite that can show this element together with all its children elements and effects.
- The SceneEffect interface which is now empty, will contain a single method - Sprite applyTo(Sprite source);
Clipping and color transformations will be removed from ElementHelpers and made effects. ClipEffect and ColorEffect classes will be created. - SceneHelper.paint(Graphics2D, ImmRect, Scene) which draws a given scene to a Graphics2D will actually get the final sprite of the scene's root element and draw it. The drawing method will be in the Sprite class.
- When the scene is redrawn, most of the sprites created will be identical and most of the tiles will be the same. Therefore it is a good idea to cache tiles. In order to do this we have to figure out a strong hashing function which doesn't depend on the pixels of the tile, but on its sprite and position.
- A class Hasher and an inner class Hash will be created.
- The Hasher accumulates the hash value of a composite object. It provides methods for adding all primitive types (int, long, char, float, double, boolean), String, Class and an already hashed object and a method to get the currently accumulated value.
- It actually applies a standart polynomial hash function several times (with different bases).
- A Hash object consists of the hash codes produced by these functions and the number of entries that have been hashed.
- An interface Hashable with a single method Hash getHash() will be created.
- Sprite will implement Hashable and all its subclasses will override the getHash method.
- ImmColor, ImmArea and some other classes will have to implement Hashable as well, so that we can add a color, area, etc. to the state of a sprite.
- A class QueryCache and an abstract inner class Query will be created.
- The Query has abstract methods for calculating the key to search for and calculating the result if no entry with the specified key is found.
- The QueryCache internally stores its entries in a Map (maybe a LinkedHashMap, so that it has a fixed size and old and unused entries expire and are removed). It has a method answering queries.
- A class Hasher and an inner class Hash will be created.
- In Sprite there will be a global QueryCache for tiles. Each getTile(TilePos) method will actually create and execute a new query. The key will be a composite hash of the sprite executing the query and the position of the wanted tile.
- http://sophie2.org/trac/browser/branches/private/deni/scenes
Hardware part: So.. I researched several possible libraries which could utilize OpenGL and video hardware. Here's an overview of all alternatives and explanations for the chosen library.
- JavaFX
- I tried it by installing a Sun's JDK bundled with JavaFX from Sun site, so installation is quite straight-forward.
- JavaFX provides a scene-oriented model, which is nice, though I don't think that integration with our Scenes model will be so easy.
- The scenes which JavaFX renders are nice looking, sleek and fast rendered but JavaFX is still in alpha and the Scenes are buggy.
- JavaFX uses its own scripting language, JavaFXScript, to define the scenes and this makes it harder to integrate with our Scenes model.
- Using such a scripting language means that defining scenes is declarative and more static than our Scenes model.
- Interaction is a bit of an issue with JavaFX scenes I guess, though I didn't test it thoroughly.
- Conclusion: JavaFX will become a very strong library in the future, I guess, though now it's not mature enough and is too different from our Scenes model, so we won't use it.
- Java3D
- Installing Java3D was a pain in the ass. It's heavily platform dependent, I couldn't find working maven repos, so maybe we can consider placing the needed JARs in our Nexus. Java3D requires some .so files under Linux to be included in the build path, so that's another problem which needs consistent solution in the future. Our installer ideas will help solve this and other similar issues.
- Java3D offers a complex and interesting 3D OO model for setting up a scene. It's quite powerful but also too different from our Scenes model.
- Integration with Java3D will be hard because:
- Java3D offers a declarative way of setting up a 3D scene. Our Scenes model is not declarative. It is imperative and the initial logic for setting up the model is quite simple but the drawing logic is the complex one, because it performs drawing by z-order to create the overlapping effect and is heavily focused of 2D drawing.
- Big parallel hierarchy of translators between our Scenes model and a Java3D world must be constructed which will be a hard and long task.
- In order to use Java3D, the Sophie book will have to be redone in a 3D fashion with pages and frames actually being rectangles in a 3D space with carefully calculated z-coordinates. It can be done, it's cool, but it requires much effort, I guess.
- Advantages of Java3D are:
- An object model which lets a developer who doesn't know much about OpenGL, to integrate new stuff by simple 3D thinking, instead of wondering about color format, long and complex OpenGL routines, etc.
- Ability to to hardware optimizations by for instance specifying in which memory a texture is being held, but again this requires in-depth OpenGL knowledge.
- I'm not quite sure about that, but Java3D seems to be a bit abandoned, or at least not so heavily used like other libs.
- JOGL
- Installing JOGL is the same pain in the ass like Java3D and it is also heavily platform dependent. Same issues apply.
- JOGL offers nothing like an OO model but it is simply a 1-to-1 wrapper library of OpenGL API C functions. Most of its code is actually autogenerated from the OpenGL API. This means that for each new version of the OpenGL API, a new JOGL version is easily updated.
- This imperative model of JOGL makes it more compatible with our Scenes model because:
- Only the painting routines need to be redone to use JOGL painting objects which are very few and easily used. This means that there will be some changes in the SceneHelpers logic to not use Graphics2D but GLDrawables.
- JOGL has some components like GLJPanel which is a hardware-accelerated JPanel which makes it very compatible with Swing.
- Transformations can be easily done by extending ImmMatrix a little and using OpenGLs built in transformation power.
- OpenGL Shaders can be pre-written in GLSL, compiled and linked runtime by JOGL and used for beautiful and very fast effects for each SceneEffect we require.
- JOGL has been developed actively in the recent times which makes it a reliable lib to use. It has lots of demos and resources and even a JavaWS Quake2 port which works perfectly. Amazing (:
- The disadvantages of JOGL are that:
- In-depth OpenGL knowledge is required because it's basically OpenGL wrapped in Java methods.
- Opposed to Java3D, writing low-level OpenGL calls is far from the ease of just specifying that you want a cube with a certain color which you want to rotate at an angle alpha around Ox at a given speed, etc.. You have to deal with vertex arrays, color formats, frustums, transformations, drawing, data representation, etc.. This is ok for a guy who knows OpenGL but otherwise it's not so good.
There are other libs like LWJGL, and others, but I guess JOGL is the lib to use because of it's list of advantages, fine control over the drawing and easiest integration with our Scenes model.
I do have a very basic working demo which is meant to be a translation of SimpleSceneDemo but I won't commit it because:
- we still don't have a strategy how to work with platform dependent JARs and .so, .dll, etc..
- using JOGL requires different compiler settings, and we need to discuss it first.
- it can't be finished until some further discussions are done.
I can show and explain the demo though, so integrators are happy (:
I also have a good concept for a complete JOGL integration design which I can do in the next revision, so I won't waste excessive time with unneeded bureaucracy by writing too much in the wiki (:
Implementation
- Algorithmic:
The current implementation cannot be integrated in trunk but some prototype code can be found here: [4419], [4434], [4477], [4488], [4564].
- Hardware:
I was agreed that the JOGL demo won't be integrated in the trunk, because of its incompleteness, manual .jar & .so installation and native-related compiler errors. I committed some version of my changes in a branch, with lots of warnings and ugly code but the demo works. In R1 of this task, there will be a nice design, more complete implementation and hopefully less build and compiler errors and problems.
The demo and related code is in [4713].
Testing
- I couldn't imagine how I should test my JOGL demo.. I have low level ideas which I've used for driver testing in NVIDIA but it's totally pointless to do so in this revision of the task.
Comments
- This task is hard and interesting (: Let's hope we'll kick ass with it.
