Leistungsprobleme bei der Verwendung von AVCaptureVideoDataOutput und AVCaptureAudioDataOutput

Question

Ich habe Verzögerungsprobleme, wenn ich Audio und Video mit AVCaptureVideoDataOutput und AVCaptureAudioDataOutput aufnehme. Manchmal blockiert das Video für einige Millisekunden, manchmal stimmt das Audio nicht mit dem Video überein.

Ich habe einige Protokolle eingefügt und beobachtet, dass ich zuerst eine Menge Videopuffer in captureOutput Callback bekomme, und nach einiger Zeit bekomme ich die Audiopuffer (manchmal bekomme ich überhaupt keine Audiopuffer, und das resultierende Video ist ohne Ton). Wenn ich den Code kommentiere, der die Videopuffer behandelt, bekomme ich die Audiopuffer ohne Probleme.

Dies ist der Code Ich verwende:

-(void)initMovieOutput:(AVCaptureSession *)captureSessionLocal 
{ 
    AVCaptureVideoDataOutput *dataOutput = [[AVCaptureVideoDataOutput alloc] init]; 
    self._videoOutput = dataOutput; 
    [dataOutput release]; 

    self._videoOutput.alwaysDiscardsLateVideoFrames = NO; 
    self._videoOutput.videoSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange] 
                  forKey:(id)kCVPixelBufferPixelFormatTypeKey 
            ];  
    AVCaptureAudioDataOutput *audioOutput = [[AVCaptureAudioDataOutput alloc] init]; 
    self._audioOutput = audioOutput; 
    [audioOutput release]; 

    [captureSessionLocal addOutput:self._videoOutput]; 
    [captureSessionLocal addOutput:self._audioOutput]; 


    // Setup the queue 
    dispatch_queue_t queue = dispatch_queue_create("MyQueue", NULL); 
    [self._videoOutput setSampleBufferDelegate:self queue:queue]; 
    [self._audioOutput setSampleBufferDelegate:self queue:queue]; 
    dispatch_release(queue); 
} 

Hier stelle ich die Schriftsteller auf:

-(BOOL) setupWriter:(NSURL *)videoURL session:(AVCaptureSession *)captureSessionLocal 
{ 
    NSError *error = nil; 
    self._videoWriter = [[AVAssetWriter alloc] initWithURL:videoURL fileType:AVFileTypeQuickTimeMovie 
                 error:&error]; 
    NSParameterAssert(self._videoWriter); 


    // Add video input 
    NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys: 
            AVVideoCodecH264, AVVideoCodecKey, 
            [NSNumber numberWithInt:640], AVVideoWidthKey, 
            [NSNumber numberWithInt:480], AVVideoHeightKey, 
            nil]; 

    self._videoWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo 
                     outputSettings:videoSettings]; 


    NSParameterAssert(self._videoWriterInput); 
    self._videoWriterInput.expectsMediaDataInRealTime = YES; 
    self._videoWriterInput.transform = [self returnOrientation]; 

    // Add the audio input 
    AudioChannelLayout acl; 
    bzero(&acl, sizeof(acl)); 
    acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono; 


    NSDictionary* audioOutputSettings = nil;   
    // Both type of audio inputs causes output video file to be corrupted. 

     // should work on any device requires more space 
     audioOutputSettings = [ NSDictionary dictionaryWithObjectsAndKeys:      
           [ NSNumber numberWithInt: kAudioFormatAppleLossless ], AVFormatIDKey, 
           [ NSNumber numberWithInt: 16 ], AVEncoderBitDepthHintKey, 
           [ NSNumber numberWithFloat: 44100.0 ], AVSampleRateKey, 
           [ NSNumber numberWithInt: 1 ], AVNumberOfChannelsKey,          
           [ NSData dataWithBytes: &acl length: sizeof(acl) ], AVChannelLayoutKey, 
           nil ]; 

    self._audioWriterInput = [AVAssetWriterInput 
             assetWriterInputWithMediaType: AVMediaTypeAudio 
             outputSettings: audioOutputSettings ]; 

    self._audioWriterInput.expectsMediaDataInRealTime = YES;  

    // add input 
    [self._videoWriter addInput:_videoWriterInput]; 
    [self._videoWriter addInput:_audioWriterInput]; 

    return YES; 
} 

Und hier wird der Rückruf:

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
     fromConnection:(AVCaptureConnection *)connection 
{ 

    if(!CMSampleBufferDataIsReady(sampleBuffer)) 
    { 
     NSLog(@"sample buffer is not ready. Skipping sample"); 
     return; 
    } 
    if(_videoWriter.status != AVAssetWriterStatusCompleted) 
    { 
     if(_videoWriter.status != AVAssetWriterStatusWriting ) 
     {    
      CMTime lastSampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer); 
      [_videoWriter startWriting]; 
      [_videoWriter startSessionAtSourceTime:lastSampleTime]; 
     } 

     if(captureOutput == _videoOutput) 
     { 
      if([self._videoWriterInput isReadyForMoreMediaData]) 
      { 

      [self newVideoSample:sampleBuffer]; 

      } 
     } 
     else if(captureOutput == _audioOutput) 
     { 
      if([self._audioWriterInput isReadyForMoreMediaData]) 
      { 

       [self newAudioSample:sampleBuffer]; 


      } 
     } 
    } 

} 

-(void) newAudioSample:(CMSampleBufferRef)sampleBuffer 
{ 

     if(_videoWriter.status > AVAssetWriterStatusWriting) 
     { 

      [self NSLogPrint:[NSString stringWithFormat:@"Audio:Warning: writer status is %d", _videoWriter.status]]; 
      if(_videoWriter.status == AVAssetWriterStatusFailed) 
       [self NSLogPrint:[NSString stringWithFormat:@"Audio:Error: %@", _videoWriter.error]]; 
      return; 
     } 

     if(![_audioWriterInput appendSampleBuffer:sampleBuffer]) 
      [self NSLogPrint:[NSString stringWithFormat:@"Unable to write to audio input"]]; 

} 

-(void) newVideoSample:(CMSampleBufferRef)sampleBuffer 
{ 
    if(_videoWriter.status > AVAssetWriterStatusWriting) 
    { 
     [self NSLogPrint:[NSString stringWithFormat:@"Video:Warning: writer status is %d", _videoWriter.status]]; 
     if(_videoWriter.status == AVAssetWriterStatusFailed) 
      [self NSLogPrint:[NSString stringWithFormat:@"Video:Error: %@", _videoWriter.error]]; 
     return; 
    } 


    if(![_videoWriterInput appendSampleBuffer:sampleBuffer]) 
     [self NSLogPrint:[NSString stringWithFormat:@"Unable to write to video input"]]; 
} 

Gibt es etwas, Falsch in meinem Code, warum verzögert sich das Video? (Ich teste es auf einem Iphone 4 ios 4.2.1)

Answer 1

Es sieht aus wie Sie serielle Warteschlangen verwenden. Die Audioausgabewarteschlange befindet sich direkt hinter der Videoausgabewarteschlange. Erwägen Sie die Verwendung von gleichzeitigen Warteschlangen.