Mobile Vision API - Concatenate New Detector Object to Continue Frame Processing

Mobile Vision API - concatenate new detector object to continue frame processing

Yes, it is possible. You'd need to create your own subclass of Detector which wraps FaceDetector and executes your extra frame processing code in the detect method. It would look something like this:

class MyFaceDetector extends Detector<Face> {
  private Detector<Face> mDelegate;

  MyFaceDetector(Detector<Face> delegate) {
    mDelegate = delegate;
  }

  public SparseArray<Face> detect(Frame frame) {
    // *** add your custom frame processing code here
    return mDelegate.detect(frame);
  }

  public boolean isOperational() {
    return mDelegate.isOperational();
  }

  public boolean setFocus(int id) {
    return mDelegate.setFocus(id);
  }
}

You'd wrap the face detector with your class, and pass your class into the camera source. It would look something like this:

    FaceDetector faceDetector = new FaceDetector.Builder(context)
            .build();
    MyFaceDetector myFaceDetector = new MyFaceDetector(faceDetector);

    myFaceDetector.setProcessor(/* include your processor here */);

    mCameraSource = new CameraSource.Builder(context, myFaceDetector)
            .build();

Your detector will be called first with the raw frame data.

Note that the image may not be upright, if the device is rotated. You can get the orientation through the frame's metadata.getRotation method.

One word of caution: once the detect method returns, you should not access the frame pixel data. Since the camera source recycles image buffers, the contents of the frame object will be eventually overridden once the method returns.

EDIT: (additional notes)
You could also avoid the boilerplate code of MyFaceDetector using a MultiDetector like this:

MultiDetector multiDetector = new MultiDetector.Builder()
    .add(new FaceDetector.Builder(context)
                .build())
    .add(new YourReallyOwnDetector())
    .build();

Also note the use of FaceTrackerFactory in conjuction with MultiProcessor
described there.

Get Frame when detecting text using a pipeline in Android Mobile Vision

The preview frame is not sent beyond the text recognizer. However, you could make a class that wraps the text recognizer, receiving the preview frame before detection. See a similar discussion here.

First, implement a detector class to wrap the text recognizer:

class MyTextRecognizer extends Detector<TextBlock> {
  private Detector<TextBlock> mDelegate;

  MyTextRecognizer(Detector<TextBlock> delegate) {
    mDelegate = delegate;
  }

  public SparseArray<TextBlock> detect(Frame frame) {
    // *** add your code to access the preview frame here
    return mDelegate.detect(frame);
  }

  public boolean isOperational() {
    return mDelegate.isOperational();
  }

  public boolean setFocus(int id) {
    return mDelegate.setFocus(id);
  }
}

Wrap the text recognizer with your class, and pass your class into the camera source. It would look something like this:

TextRecognizer textRecognizer = new TextRecognizer.Builder(context)
        .build();
TextRecognizer myTextRecognizer = new MyTextRecognizer(textRecognizer);

myTextRecognizer.setProcessor(/* include your processor here */);

mCameraSource = new CameraSource.Builder(context, myTextRecognizer)
        .build();

Your MyTextRecognizer will be called first with the raw frame data.

Note that the image may not be upright, if the device is rotated. You can get the orientation through the frame's metadata.getRotation method.

One word of caution: once the detect method returns, you should not access the frame pixel data. Since the camera source recycles image buffers, the contents of the frame object will be eventually overridden once the method returns.

Limit Detection Area in Vision API

The API currently does not have an option to limit the detection area. But you could crop the preview image before it gets passed into the barcode detector. See here for an outline of how to wrap a detector with your own class:

Mobile Vision API - concatenate new detector object to continue frame processing

You'd implement the "detect" method to take the frame received from the camera, create a cropped version of the frame, and pass that through to the underlying detector.

---

Related Topics