audio-tears-on-some-video-formats - Audio playback of some videos, goes silent periodically during playback.

[todor]

With some video files the following problem occurs:

• Insert video file.
• Open the book in preview mode.
• Start the video playback.
• Notice that the audio periodically goes quiet, and skips a little.

[meddle]

You missed the problem with the jerky video... but ok...

[meddle]

• The problems:
  • About the audio problem with some of the formats:
    • In the native media_decoder the BUFFER_EXTRA is too small so the audio which is coded to have key frames skips some of them while decoding and we can not hear it... but if we make the BUFFER_EXTRA larger the performance of the video will be low, because we read audio and video form the natives in one thread - the EventDispatchThread.
    • Note that in the trunk the video is jerky because of that too.
  • The jerky video problem:
    • We need a way to read the audio in another thread so the video and the audio to be independent...

• Solutions:
  • Add a new AudioThread nested class (that extends Thread) in the NativeMediaHandler class
  • It will have two blocking queues, one for requests, and another for responses.
  • It will have LinkedHashMap (with some CAPACITY) cache for caching prefetched AudioChunks.
  • Its main loop will do the following:
    • If there are requests in the requests queue:
      • Get request from it.
      • If there is no response for that request in the cache, retrieve it from the natives.
      • Get the response for the request from the cache and put it into the responses queue...
    • Else check if the cache needs to be filled using some PREFETCH_COUNT:
      • If the cache does not contain all the requests from the last one to the PREFETCH_COUNT request them from the natives.
      • Otherwise sleep for some time...
  • The thread will have getAudio method that works in this way:
    • If the requested time's chunk is in the cache, returns it,
    • Otherwise puts it into the requests queue and waits for a response from the responses queue, using it blocking mechanism.

• For the implementation we will have to decide how big will be the cache and the prefetch count in the java part, and the extra and step buffer in the C++ part.
• We will have to test for performance issues with the video/audio too.

[meddle]

Some patch for the design

[pap]

• Awesome

[meddle]

C++ part patch

[meddle]

C++ part patch

[meddle]

Java part patch

[meddle]

Tested on my machine (linux 32) to determine the length of the buffers...

• I use patches because it's faster... but I have two suggestions to you:
  • Delete the VideoDemo class.
  • Rename AudioDemo to MediaDemo.
• Test the videos and audios in Sophie and with the demo please...

[pap]

• Ok with still some problems with some videos on soe machines :-/
• Commited in [8526], [8527], [8528], [8529], [8530].

[meddle]

Closing all the tickets before M Y1