对于 1080p 视频，MediaCodec.configure 失败并出现 IllegalStateException

我一直在使用grafika的电影播放器 https://github.com/google/grafika/blob/master/src/com/android/grafika/MoviePlayer.java和大弗莱克的提取Mpeg帧测试 http://bigflake.com/mediacodec/ExtractMpegFramesTest_egl14.java.txt在我们的应用程序中实现搜索和提取帧功能。这些对于我们的大多数用户来说工作得很好，但是他们中的一些人遇到了IllegalStateException on MediaCodec.configure设置时ExtractMpegFramesTest（这种情况发生在三星 Galaxy S4 mini、三星 Galaxy J7、三星 Galaxy A5、华为 Ascend G7 上）。相关代码如下：

MoviePlayer

public void prepareResources() throws IOException {
    // The MediaExtractor error messages aren't very useful.  Check to see if the input
    // file exists so we can throw a better one if it's not there.
    if (!mSourceFile.canRead()) {
        throw new FileNotFoundException("Unable to read " + mSourceFile);
    }

    try {
        mExtractor = new MediaExtractor();
        mExtractor.setDataSource(mSourceFile.toString());
        mTrackIndex = selectTrack(mExtractor);
        if (mTrackIndex < 0) {
            throw new RuntimeException("No video track found in " + mSourceFile);
        }
        mExtractor.selectTrack(mTrackIndex);

        MediaFormat format = mExtractor.getTrackFormat(mTrackIndex);

        // Create a MediaCodec decoder, and configure it with the MediaFormat from the
        // extractor.  It's very important to use the format from the extractor because
        // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
        String mime = format.getString(MediaFormat.KEY_MIME);
        mDecoder = MediaCodec.createDecoderByType(mime);
        mDecoder.configure(format, mOutputSurface, null, 0);
        mDecoder.start();
        mDecoderInputBuffers = mDecoder.getInputBuffers();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

ExtractMpegFramesTest

private void extractMpegFrames() throws IOException {
    MediaCodec decoder = null;
    CodecOutputSurface outputSurface = null;
    MediaExtractor extractor = null;
    /*int saveWidth = 640;
    int saveHeight = 480;*/

    try {
        long start1 = System.currentTimeMillis();

        File inputFile = new File(mInputVideoPath);   // must be an absolute path
        // The MediaExtractor error messages aren't very useful.  Check to see if the input
        // file exists so we can throw a better one if it's not there.
        if (!inputFile.canRead()) {
            throw new FileNotFoundException("Unable to read " + inputFile);
        }

        extractor = new MediaExtractor();
        extractor.setDataSource(inputFile.toString());
        int trackIndex = selectTrack(extractor);
        if (trackIndex < 0) {
            throw new RuntimeException("No video track found in " + inputFile);
        }
        extractor.selectTrack(trackIndex);

        MediaFormat format = extractor.getTrackFormat(trackIndex);
        if (VERBOSE) {
            Log.d(TAG, "Video size is " + format.getInteger(MediaFormat.KEY_WIDTH) + "x" +
                    format.getInteger(MediaFormat.KEY_HEIGHT));
        }

        // Could use width/height from the MediaFormat to get full-size frames.
        outputSurface = new CodecOutputSurface(format.getInteger(MediaFormat.KEY_WIDTH), format.getInteger(MediaFormat.KEY_HEIGHT));

        // Create a MediaCodec decoder, and configure it with the MediaFormat from the
        // extractor.  It's very important to use the format from the extractor because
        // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
        String mime = format.getString(MediaFormat.KEY_MIME);
        decoder = MediaCodec.createDecoderByType(mime);
        decoder.configure(format, outputSurface.getSurface(), null, 0); //fails right here
        decoder.start();

        long end1 = System.currentTimeMillis();
        Timber.d("extractMpegFrames(): FRAME_EXTRACT setup in: %d millis", (end1-start1));

        long start2 = System.currentTimeMillis();
        doExtract(extractor, trackIndex, decoder, outputSurface);
        long end2 = System.currentTimeMillis();
        Timber.d("extractMpegFrames(): FRAME_EXTRACT doExtract in: %d millis", (end2-start2));
    } finally {
        // release everything we grabbed
        if (outputSurface != null) {
            outputSurface.release();
            outputSurface = null;
        }
        if (decoder != null) {
            decoder.stop();
            decoder.release();
            decoder = null;
        }
        if (extractor != null) {
            extractor.release();
            extractor = null;
        }
    }
}

我知道这一点question https://stackoverflow.com/a/17243175/1602807，但我不明白的是为什么MediaCodec可以配置确定MoviePlayer（视频运行得很好）但失败了ExtractMpegFramesTest。起初，我认为设备的 OpenGL 设置存在一些问题，但结果是 MediaCodec 问题。

任何见解将不胜感激。

EDIT:获得 Galaxy S4 mini 测试设备后，我获得了更有用的日志：

D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
D/MoviePlayer: Video size is 1920x1080
D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
I/OMXClient: Using client-side OMX mux.
E/ACodec: [OMX.qcom.video.decoder.avc] storeMetaDataInBuffers failed w/ err -2147483648
E/ACodec:  configureCodec multi window instance fail  appPid : 3283
E/ACodec: [OMX.qcom.video.decoder.avc] configureCodec returning error -38
E/MediaCodec: Codec reported an error. (omx error 0x80001001, internalError -38)
W/System.err: java.lang.IllegalStateException
W/System.err:     at android.media.MediaCodec.native_configure(Native Method)
W/System.err:     at android.media.MediaCodec.configure(MediaCodec.java:262)
W/System.err:     at co.test.testing.player.MoviePlayer.prepareResources(MoviePlayer.java:237)
W/System.err:     at co.test.testing.activities.VideoActivity.surfaceCreated(VideoActivity.java:276)
W/System.err:     at android.view.SurfaceView.updateWindow(SurfaceView.java:602)
W/System.err:     at android.view.SurfaceView.access$000(SurfaceView.java:94)
W/System.err:     at android.view.SurfaceView$3.onPreDraw(SurfaceView.java:183)
W/System.err:     at android.view.ViewTreeObserver.dispatchOnPreDraw(ViewTreeObserver.java:891)
W/System.err:     at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2201)
W/System.err:     at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1256)
W/System.err:     at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:6635)
W/System.err:     at android.view.Choreographer$CallbackRecord.run(Choreographer.java:813)
W/System.err:     at android.view.Choreographer.doCallbacks(Choreographer.java:613)
W/System.err:     at android.view.Choreographer.doFrame(Choreographer.java:583)
W/System.err:     at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:799)
W/System.err:     at android.os.Handler.handleCallback(Handler.java:733)
W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:95)
W/System.err:     at android.os.Looper.loop(Looper.java:146)
W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:5593)
W/System.err:     at java.lang.reflect.Method.invokeNative(Native Method)
W/System.err:     at java.lang.reflect.Method.invoke(Method.java:515)
W/System.err:     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
W/System.err:     at dalvik.system.NativeStart.main(Native Method)

这些是从有此问题的设备中检索到的编解码器信息：

• OMX.qcom.video.encoder.avc

• OMX.SEC.avc.enc

• OMX.Exynos.AVC.编码器

奇怪的是 Galaxy S4 mini 可以很好地处理 720p 视频，只是处理 1080p 视频时出现问题。

好吧，在这里再次回答我自己的问题：

• 首先，为什么我无法提取帧ExtractMpegFramesTest使用后MoviePlayer显示视频：

  某些设备似乎无法处理 2 个实例MediaCodec同时对于高分辨率视频（对于 Galaxy S4 mini，>720p），因此您必须正确释放一个实例MediaCodec在开始另一项之前，像这样：

  public void releaseResources() {
      // release everything we grabbed
      if (mDecoder != null) {
          try {
              mDecoder.stop();
              mDecoder.release();
              mDecoder = null;
          } catch (Exception e) {
              Timber.d("releaseResources(): message %s cause %s", e.getMessage(),  e.getCause());
              e.printStackTrace();
          }
      }
      if (mExtractor != null) {
          try {
              mExtractor.release();
              mExtractor = null;
          } catch (Exception e) {
              Timber.d("releaseResources(): message %s cause %s", e.getMessage(),  e.getCause());
              e.printStackTrace();
          }
      }
  }
  

• 其次，为什么MoviePlayer无法配置 1080p 视频。就我而言，因为我从媒体选择器活动开始此活动，VideoView用于预览，其中有一个MediaPlayer.OnErrorListener()注册以检测有缺陷的视频。问题是onError()回调在 Galaxy S4 mini 上有一些非常奇怪的行为，而它在我的开发设备上工作得很好。我的疯狂猜测MediaPlayer.OnErrorListener()留下MediaCodec状态不好，以后会引起很多头痛。

  所以解决办法就是设置一个空白OnErrorListener()并正确释放VideoView在做任何其他事情之前MediaCodec。像这样的东西：

  mVideoView.setOnErrorListener(new MediaPlayer.OnErrorListener() {
          @Override
          public boolean onError(MediaPlayer mp, int what, int extra) {
              return true;
          }
  });
  .....
  mVideoView.stopPlayback();
  doOtherThingWithMediaCodec();
  

这解释了在不同设备上发生的许多奇怪的事情。我希望这可以帮助其他人调试他们的应用程序。