Ticket #2259 (closed bug: obsolete)

Opened 10 years ago

Last modified 8 years ago

audio-tears-on-some-video-formats - Audio playback of some videos, goes silent periodically during playback.

Reported by: todor Owned by: meddle
Priority: major Milestone: X3
Component: uncategorized Version: 2.0
Keywords: Cc:
Category: unknown Effort:
Importance: 86 Ticket_group:
Estimated Number of Hours: 0 Add Hours to Ticket: 0
Billable?: yes Total Hours: 0
Analysis_owners: todor Design_owners: milo, meddle
Imp._owners: milo, meddle Test_owners:
Analysis_reviewers: Changelog:
Design_reviewers: pap Imp._reviewers: pap
Test_reviewers: Analysis_score: 3.5
Design_score: 4.5 Imp._score: 4
Test_score: 0

Description

With some video files the following problem occurs:

  • Insert video file.
  • Open the book in preview mode.
  • Start the video playback.
  • Notice that the audio periodically goes quiet, and skips a little.

Attachments

2259.patch (13.7 KB) - added by meddle 10 years ago.
Some patch for the design
2259_imp_natives.patch (729 bytes) - added by meddle 10 years ago.
C++ part patch
2259_imp_natives.2.patch (729 bytes) - added by meddle 10 years ago.
C++ part patch
2259_imp_java.patch (13.7 KB) - added by meddle 10 years ago.
Java part patch

Change History

comment:1 Changed 10 years ago by todor

  • Status changed from new to s1b_analysis_finished

comment:2 Changed 10 years ago by deyan

  • Importance set to 86

comment:3 Changed 10 years ago by meddle

  • Design_owners set to meddle
  • Status changed from s1b_analysis_finished to s2a_design_started
  • Analysis_score changed from 0 to 3.5

You missed the problem with the jerky video... but ok...

comment:4 Changed 10 years ago by meddle

  • The problems:
    • About the audio problem with some of the formats:
      • In the native media_decoder the BUFFER_EXTRA is too small so the audio which is coded to have key frames skips some of them while decoding and we can not hear it... but if we make the BUFFER_EXTRA larger the performance of the video will be low, because we read audio and video form the natives in one thread - the EventDispatchThread.
      • Note that in the trunk the video is jerky because of that too.
    • The jerky video problem:
      • We need a way to read the audio in another thread so the video and the audio to be independent...
  • Solutions:
    • Add a new AudioThread nested class (that extends Thread) in the NativeMediaHandler class
    • It will have two blocking queues, one for requests, and another for responses.
    • It will have LinkedHashMap (with some CAPACITY) cache for caching prefetched AudioChunks.
    • Its main loop will do the following:
      • If there are requests in the requests queue:
        • Get request from it.
        • If there is no response for that request in the cache, retrieve it from the natives.
        • Get the response for the request from the cache and put it into the responses queue...
      • Else check if the cache needs to be filled using some PREFETCH_COUNT:
        • If the cache does not contain all the requests from the last one to the PREFETCH_COUNT request them from the natives.
        • Otherwise sleep for some time...
    • The thread will have getAudio method that works in this way:
      • If the requested time's chunk is in the cache, returns it,
      • Otherwise puts it into the requests queue and waits for a response from the responses queue, using it blocking mechanism.
  • For the implementation we will have to decide how big will be the cache and the prefetch count in the java part, and the extra and step buffer in the C++ part.
  • We will have to test for performance issues with the video/audio too.

Changed 10 years ago by meddle

Some patch for the design

comment:5 Changed 10 years ago by meddle

  • Design_owners changed from meddle to milo, meddle
  • Status changed from s2a_design_started to s2b_design_finished

comment:6 Changed 10 years ago by pap

  • Status changed from s2b_design_finished to s2c_design_ok
  • Design_score changed from 0 to 4.5
  • Design_reviewers set to pap
  • Awesome

comment:7 Changed 10 years ago by meddle

  • Owner set to meddle
  • Status changed from s2c_design_ok to s3a_implementation_started
  • Imp._owners set to milo, meddle

Changed 10 years ago by meddle

C++ part patch

Changed 10 years ago by meddle

C++ part patch

Changed 10 years ago by meddle

Java part patch

comment:8 Changed 10 years ago by meddle

  • Status changed from s3a_implementation_started to s3b_implementation_finished

Tested on my machine (linux 32) to determine the length of the buffers...

  • I use patches because it's faster... but I have two suggestions to you:
    • Delete the VideoDemo class.
    • Rename AudioDemo to MediaDemo.
  • Test the videos and audios in Sophie and with the demo please...

comment:9 Changed 10 years ago by pap

  • Status changed from s3b_implementation_finished to s3c_implementation_ok
  • Imp._score changed from 0 to 4
  • Imp._reviewers set to pap

comment:10 Changed 8 years ago by meddle

  • Status changed from s3c_implementation_ok to closed
  • Resolution set to obsolete

Closing all the tickets before M Y1

Note: See TracTickets for help on using tickets.