Performs image segmentation on images.
Note that, in addition to the standard segmentation API, ERROR(/segment)
and ERROR(/segmentForVideo)
, that take an input image and return the outputs, but involves deep copy of the
returns, ImageSegmenter also supports the callback API, ERROR(/segmentWithResultListener)
and
ERROR(/segmentForVideoWithResultListener)
, which allow you to access the outputs through zero
copy.
The callback API is available for all RunningMode
in ImageSegmenter. Set OutputHandler.ResultListener
in ImageSegmenter.ImageSegmenterOptions
properly to use the callback API.
The API expects a TFLite model with,TFLite Model Metadata..
- Input image
MPImage
- The image that image segmenter runs on.
- Output ImageSegmenterResult
ImageSegmenterResult
- An ImageSegmenterResult containing segmented masks.
Nested Classes
class | ImageSegmenter.ImageSegmenterOptions | Options for setting up an ImageSegmenter . |
|
class | ImageSegmenter.SegmentationOptions | Options for configuring runtime behavior of ImageSegmenter . |
Public Methods
static ImageSegmenter |
createFromOptions(Context context, ImageSegmenter.ImageSegmenterOptions segmenterOptions)
Creates an
ImageSegmenter instance from an ImageSegmenter.ImageSegmenterOptions . |
List<String> |
getLabels()
Get the category label list of the ImageSegmenter can recognize.
|
ImageSegmenterResult |
segment(MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions)
Performs image segmentation on the provided single image.
|
ImageSegmenterResult |
segment(MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs image segmentation on the provided single image.
|
ImageSegmenterResult | |
void |
segmentAsync(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Sends live image data to perform image segmentation, and the results will be available via the
OutputHandler.ResultListener provided in the ImageSegmenter.ImageSegmenterOptions . |
void |
segmentAsync(MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions, long timestampMs)
Sends live image data to perform image segmentation, and the results will be available via the
OutputHandler.ResultListener provided in the ImageSegmenter.ImageSegmenterOptions . |
void |
segmentAsync(MPImage image, long timestampMs)
Sends live image data to perform image segmentation with default image processing options, i.e.
|
ImageSegmenterResult |
segmentForVideo(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs image segmentation on the provided video frame.
|
ImageSegmenterResult |
segmentForVideo(MPImage image, long timestampMs)
Performs image segmentation on the provided video frame with default image processing options,
i.e.
|
ImageSegmenterResult |
segmentForVideo(MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions, long timestampMs)
Performs image segmentation on the provided video frame.
|
void |
segmentForVideoWithResultListener(MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs image segmentation on the provided video frame, and provides zero-copied results via
OutputHandler.ResultListener in ImageSegmenter.ImageSegmenterOptions . |
void |
segmentForVideoWithResultListener(MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions, long timestampMs)
Performs image segmentation on the provided video frame, and provides zero-copied results via
OutputHandler.ResultListener in ImageSegmenter.ImageSegmenterOptions . |
void |
segmentForVideoWithResultListener(MPImage image, long timestampMs)
Performs image segmentation on the provided video frame with default image processing options,
i.e.
|
void |
segmentWithResultListener(MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs image segmentation on the provided single image, and provides zero-copied results via
OutputHandler.ResultListener in ImageSegmenter.ImageSegmenterOptions . |
void |
segmentWithResultListener(MPImage image)
Performs image segmentation on the provided single image with default image processing options,
i.e.
|
void |
segmentWithResultListener(MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions)
Performs image segmentation on the provided single image, and provides zero-copied results via
OutputHandler.ResultListener in ImageSegmenter.ImageSegmenterOptions . |
Inherited Methods
Public Methods
public static ImageSegmenter createFromOptions (Context context, ImageSegmenter.ImageSegmenterOptions segmenterOptions)
Creates an ImageSegmenter
instance from an ImageSegmenter.ImageSegmenterOptions
.
Parameters
context | an Android ERROR(/Context) . |
---|---|
segmenterOptions | an ImageSegmenter.ImageSegmenterOptions instance. |
Throws
if there is an error during ImageSegmenter creation.
|
public List<String> getLabels ()
Get the category label list of the ImageSegmenter can recognize. For CATEGORY_MASK type, the index in the category mask corresponds to the category in the label list. For CONFIDENCE_MASK type, the output mask list at index corresponds to the category in the label list.
If there is no labelmap provided in the model file, empty label list is returned.
public ImageSegmenterResult segment (MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions)
Performs image segmentation on the provided single image. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.IMAGE)
. TODO update java doc
for input image format.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
segmentationOptions | the ImageSegmenter.SegmentationOptions used to configure the runtime
behavior of the ImageSegmenter . |
Throws
if there is an internal error. Or if ImageSegmenter is
created with a OutputHandler.ResultListener .
|
public ImageSegmenterResult segment (MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs image segmentation on the provided single image. The output mask has the same size as
the input image. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.IMAGE)
. TODO update java doc for input image format.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. Or if ImageSegmenter is
created with a OutputHandler.ResultListener .
|
public ImageSegmenterResult segment (MPImage image)
Performs image segmentation on the provided single image with default image processing options,
i.e. without any rotation applied. The output mask has the same size as the input image. Only
use this method when the ImageSegmenter
is created with ERROR(/RunningMode.IMAGE)
.
TODO update java doc for input image format.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|
Throws
if there is an internal error. Or if ImageSegmenter is
created with a OutputHandler.ResultListener .
|
public void segmentAsync (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Sends live image data to perform image segmentation, and the results will be available via the
OutputHandler.ResultListener
provided in the ImageSegmenter.ImageSegmenterOptions
. The output mask has the
same size as the input image. Only use this method when the ImageSegmenter
is created
with ERROR(/RunningMode.LIVE_STREAM)
.
It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the image segmenter. The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
timestampMs | the input timestamp (in milliseconds). |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. |
public void segmentAsync (MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions, long timestampMs)
Sends live image data to perform image segmentation, and the results will be available via the
OutputHandler.ResultListener
provided in the ImageSegmenter.ImageSegmenterOptions
. Only use this method when
the ImageSegmenter
is created with ERROR(/RunningMode.LIVE_STREAM)
.
It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the image segmenter. The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
segmentationOptions | the ImageSegmenter.SegmentationOptions used to configure the runtime
behavior of the ImageSegmenter . |
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. |
public void segmentAsync (MPImage image, long timestampMs)
Sends live image data to perform image segmentation with default image processing options, i.e.
without any rotation applied, and the results will be available via the OutputHandler.ResultListener
provided in the ImageSegmenter.ImageSegmenterOptions
. The output mask has the same size as the input
image. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.LIVE_STREAM)
.
It's required to provide a timestamp (in milliseconds) to indicate when the input image is sent to the image segmenter. The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. |
public ImageSegmenterResult segmentForVideo (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs image segmentation on the provided video frame. The output mask has the same size as
the input image. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
timestampMs | the input timestamp (in milliseconds). |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. Or if ImageSegmenter is
created with a OutputHandler.ResultListener .
|
public ImageSegmenterResult segmentForVideo (MPImage image, long timestampMs)
Performs image segmentation on the provided video frame with default image processing options,
i.e. without any rotation applied. The output mask has the same size as the input image. Only
use this method when the ImageSegmenter
is created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. Or if ImageSegmenter is
created with a OutputHandler.ResultListener .
|
public ImageSegmenterResult segmentForVideo (MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions, long timestampMs)
Performs image segmentation on the provided video frame. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
segmentationOptions | the ImageSegmenter.SegmentationOptions used to configure the runtime
behavior of the ImageSegmenter . |
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. Or if ImageSegmenter is
created with a OutputHandler.ResultListener .
|
public void segmentForVideoWithResultListener (MPImage image, ImageProcessingOptions imageProcessingOptions, long timestampMs)
Performs image segmentation on the provided video frame, and provides zero-copied results via
OutputHandler.ResultListener
in ImageSegmenter.ImageSegmenterOptions
. The output mask has the same size as
the input image. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | |
timestampMs | the input timestamp (in milliseconds). |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. Or if ImageSegmenter is not
created with OutputHandler.ResultListener set in ImageSegmenter.ImageSegmenterOptions .
|
public void segmentForVideoWithResultListener (MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions, long timestampMs)
Performs image segmentation on the provided video frame, and provides zero-copied results via
OutputHandler.ResultListener
in ImageSegmenter.ImageSegmenterOptions
. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
segmentationOptions | the ImageSegmenter.SegmentationOptions used to configure the runtime
behavior of the ImageSegmenter . |
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. Or if ImageSegmenter is not
created with OutputHandler.ResultListener set in ImageSegmenter.ImageSegmenterOptions .
|
public void segmentForVideoWithResultListener (MPImage image, long timestampMs)
Performs image segmentation on the provided video frame with default image processing options,
i.e. without any rotation applied, and provides zero-copied results via OutputHandler.ResultListener
in ImageSegmenter.ImageSegmenterOptions
. The output mask has the same size as the input image. Only
use this method when the ImageSegmenter
is created with ERROR(/RunningMode.VIDEO)
.
It's required to provide the video frame's timestamp (in milliseconds). The input timestamps must be monotonically increasing.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
timestampMs | the input timestamp (in milliseconds). |
Throws
if there is an internal error. Or if ImageSegmenter is not
created with OutputHandler.ResultListener set in ImageSegmenter.ImageSegmenterOptions .
|
public void segmentWithResultListener (MPImage image, ImageProcessingOptions imageProcessingOptions)
Performs image segmentation on the provided single image, and provides zero-copied results via
OutputHandler.ResultListener
in ImageSegmenter.ImageSegmenterOptions
. The output mask has the same size as
the input image. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.IMAGE)
.
TODO update java doc for input image format.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
imageProcessingOptions | the ImageProcessingOptions specifying how to process the
input image before running inference. Note that region-of-interest is not supported
by this task: specifying ImageProcessingOptions.regionOfInterest() will result in
this method throwing an IllegalArgumentException. |
Throws
IllegalArgumentException | if the ImageProcessingOptions specify a
region-of-interest. |
---|---|
if there is an internal error. Or if ImageSegmenter is not
created with OutputHandler.ResultListener set in ImageSegmenter.ImageSegmenterOptions .
|
public void segmentWithResultListener (MPImage image)
Performs image segmentation on the provided single image with default image processing options,
i.e. without any rotation applied, and provides zero-copied results via OutputHandler.ResultListener
in ImageSegmenter.ImageSegmenterOptions
. The output mask has the same size as the input image. Only
use this method when the ImageSegmenter
is created with ERROR(/RunningMode.IMAGE)
.
TODO update java doc for input image format.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|
Throws
if there is an internal error. Or if ImageSegmenter is not
created with OutputHandler.ResultListener set in ImageSegmenter.ImageSegmenterOptions .
|
public void segmentWithResultListener (MPImage image, ImageSegmenter.SegmentationOptions segmentationOptions)
Performs image segmentation on the provided single image, and provides zero-copied results via
OutputHandler.ResultListener
in ImageSegmenter.ImageSegmenterOptions
. Only use this method when the ImageSegmenter
is created with ERROR(/RunningMode.IMAGE)
.
TODO update java doc for input image format.
ImageSegmenter
supports the following color space types:
Parameters
image | a MediaPipe MPImage object for processing. |
---|---|
segmentationOptions | the ImageSegmenter.SegmentationOptions used to configure the runtime
behavior of the ImageSegmenter . |
Throws
if there is an internal error. Or if ImageSegmenter is not
created with OutputHandler.ResultListener set in ImageSegmenter.ImageSegmenterOptions .
|