Error: Macro BackLinksMenu(None) failed

compressed data is corrupt

Error: Macro TicketQuery(summary=GROUP_APP_PERFORMANCE_R1, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed

current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

The goal of this task is to improve the performance of the Sophie2 platform before the weekly at 2009-05-26. Concrete subtasks in the next section.

Task requirements

• A custom profiler should be developed for our needs. Lots of profilers exist, I tried some, but for now I think a simple profiler can be written quickly and have some features which I couldn't find in other profilers. So what should the custom profiler do:
  • Measure time consumption per method as a percentage of the overall time.
    • Provide an own mode of measurement -- this means to measure the time which the method itself consumed, excluding the time any other methods that were called consumed.
    • Provide a total mode of measurement -- this includes the time which other invoked methods consumed.
  • Sort the above results by decreasing order of percentage of the total time.
    • This includes truncating the result to only the first N (given some N). Such reports tend to waste much memory and usually only some of the first ones are interesting and useful for being targeted for optimization.
  • Structure the profiler report by the natural stack trace execution tree-like structure. This means that at the top of the tree there should be the main() run method of FakeAuthorMain for instance with total time consumption of 100%. Lower there should be methods which are directly invoked from main() and the sum of their total time consumption should be 100%, etc.
  • This profiler is intended to locate higher level methods which consume too much time and/or are called too many times, so we can optimize them. Locating that lowest level methods like native library file system IO calls are called lots of times doesn't help much because we can't do much to optimize such cases, but finding out that ReflectionUtil.findMethodAnnotation() consumes much time led to a big startup performance optimization before.
  • If other useful features come up in mind and can be implemented in time, implement them.
  • Provide a report with the described results in an output plain-text file.
  • Providing the functionality described above is generally done in two ways -- by actually measuring real execution time and by statistically measuring time consumption. The second is done by doing snapshots of the program's state at a given interval and gathering statistics about current methods in all alive threads. First one seems to be too slow and me and Milo think that the second gives a good enough accuracy of the result, so use the second approach.

• If a good enough profiler is proposed and there's no longer a need for a customer profiler, drop it and use the existing one.

• Use the (custom or already-existing) profiler to identify low performance spots which could be optimized.

• Optimize these spots by either doing quick low-level hacks or by proposing new design which could be implemented in time.

Task result

• A useful profiler should be developed/chosen and a how-to-use section should be included in the wiki.
• Ideas for optimizations should be proposed for concrete low performance spots.
• If there's time left, implement optimizations.

Implementation idea

• hprof's file format can be used as a base for the output format of our custom profiler.
• Use Thread.getAllStackTraces() and then build a tree.

Related

• GROUP_APP_PERFORMANCE_R0
• GROUP_PRO_LIB_PERFORMANCE_R0
• GROUP_PRO_LIB_PERFORMANCE_R1

How to demo

^{(Provide instructions for demonstration of the task.)}

Design

Implementation

Testing

^{(Place the testing results here.)}

Comments

^{(Write comments for this or later revisions here.)}