Multitasking Essentials for Media-Based Apps on iPad in iOS 9

Picture in Picture(PiP) is one of killer feature in iOS 9.

Great article about multitasking

Related session : Introducing AVKit in iOS 9 WWDC14 , Mastering modern media playback

Deprecated APIs in iOS 9

• MPMoviePlayerController
• MPMoviePlayerViewController

Replaced by AVPlayerViewController

Support PiP frameworks

• AVKit
  • AVPlayerViewController
  • Set apps background mode
    • Audio and airplay checked
    • AVAudioSession.sharedInstance() AudioSession Catogory setAVAudioSessionCategoryPlayback
    • allowsPictureinPicture in AVPlayerViewController if you want to stop PiP
    • Demo project : AVFoundationPiPPlayer: Picture-in-Picture Playback with AVKit

• AVFoundation

  • AVPictureInPictureController for custom player controls

    // Check whether Picture in Picture is supported on device.
    if AVPictureInPictureController.isPictureInPictureSupported() {
      // Create Picture in Picture controller.
      pipController = AVPictureInPictureController(playerLayer: playerLayer)!
      // Set delegate.
      pipController.delegate = self
    }
    //
    // Find out whether Picture in Picture is possible.
    let pipPossible = pipController.pictureInPicturePossible
    // Enable/disable Picture in Picture button.
    pipButton.enabled = pipPossible
    //
    func pipButtonTapped(sender: AnyObject?) {
      // Make sure Picture in Picture is not already active.
      if !pipController.pictureInPictureActive {
          // Start Picture in Picture on button tap.
          pipController.startPictureInPicture()
      }
    }
    //Delegate
    func pictureInPictureControllerDidStartPictureInPicture(pipController:
    AVPictureInPictureController) {
      // Dismiss modal video playback view controller.
      dismissViewControllerAnimated(true, completion: nil)
    }
    func pictureInPictureController(pipController: AVPictureInPictureController,
    restoreUserInterfaceForPictureInPictureStopWithCompletionHandler
    completionHandler: (Bool) -> Void) {
      // Present video playback view controller again.
      navigationController?.presentViewController(self, animated: true)     {
      // Don’t forget to call completion handler.
      completionHandler(true)
      }
    }
    

  • func pictureInPictureControllerWillStartPictureInPicture(pipController: AVPictureInPictureController) will be called when pip is about to starty
    • hide player control as the controls on pip
    • show placeholder to indicate the media is shown in pip

  • func pictureInPictureControllerDidStopPictureInPicture(pipController: AVPictureInPictureController)

    • show player control again as pip is dismissed
    • hide the placeholder to show media

  • Demo Project : AVFounationPiPPlayer

• Webkit
  • WKWebView supports background modes
  • WKWebConfiguarion to config allowsPictureinPicture default is yes
  • Related session : What’s New in Web Developers in WebKit and Safari
• PiP almost same as background audio
  • reject if any violation of rules

Best Practices

• iPad is now have many foreground apps
  • full screen to serve only primary app
  • slide over will deactivite the primary and show secondary app temporary
  • split view to show 2 apps on a screen
  • PiP treats as background media
  • How to handle different situations for shared resources like audio, video and camera
  • Audio
    • Use one configuration
    • Only activate session when audio is first needed
    • Use Ambient category for game and sound effect and it is not interrupt the audio player underlying
    • use secondaryAudioShouldBeSilencedHint to check if secondary audio shuld be silenced
    • Related session What’s New in Core Audio in WWDC 2014
    • Audio Session Programming Guide on developer.apple.com/iOS
  • Video
    • use different variant of videos (high-res for full screen and low-Res for PiP)
  • Camera
    • One app can use camera
      • Availability can charge at any time.
      • UIRequiresFullscreen = YES if requires whole screen as the camera view finder
      • UIImagePickerController
        • 
        • use startViewCapture() return value to see if video capturing is available
        • 
      • AVCaptureSession
        • Observe AVCaptureSessionWasInterruptedNotification for interruption
        • AVCaptureSessionInterruptionReasonKey would give VideoDeviceNotAvailableWithMultipleForegroundApps if other app takes camera control
        • Adjust UI accordingly if interruption is happened.
        • After interruption is completed, AVCaptureSession will be resumes automatically
        • AVCaptureSessionInterruptionEndedNotification for listening the interruption is ended
        • Demo Project : AVCam for more details