Facemesh from mediapipe #1316
Facemesh from mediapipe #1316
Conversation
There was a problem hiding this comment.
Hey @David-rn,
Excellent work as always! There's a few extra tests in the Colab that I'd like to see.
- I see the section "Modern" at the end, but the code is missing; likely not saved. This is probably the copy of what's in the docstring + download of the landmarker task.
- Can we test with an empty image?
np.empty((640,640,3), dtype=np.uint8)usually works. - I wonder, can this detect more than 1 face?
|
Hi @LinasKo, Thanks for the comments! My bad!! I don't know why it was not saved. In the same link you can find now the saved version with the extra considerations:
|
|
@David-rn thanks for fixing collab (I was also curious why is it was missing, I thought you were working) can you change commit sha in last commit for trigger pre-commit (There was a github server issue at the time but it is fixed so pre-commit should work fine again) Without any change you can do this |
5e94100
to
a023259
Compare
|
I've tested it with multiple images, checked out how the docs look, added a bit of a change. I believe my suggestion in the colab to test on an empty image wasn't entirely correct - I meant Thank you @David-rn (and @onuralpszr) - another flawless, well-described contribution. It's a pleasure working with you :) Merging! (fyi @SkalskiP, skeletons.py now contains a variable which is a VERY long list of edges. I want the functionality in the repo now, but if it bothers us, we can export it to a json file later). |
Description
According to issue #1274, this PR extends the usage of
from_mediapipefunction to support FaceMesh.Type of change
Please delete options that are not relevant.
How has this change been tested, please provide a testcase or example of how you tested the change?
The following Google Colab describes the process used for testing this new feature. It allows to upload a picture to test it. (This Colab feature only worked in Chrome for me).
The first part of the Colab is the examples from the PoseLandmaker to ensure that this functionality is still working after the new changes.
Any specific deployment considerations
As discussed in the issue thread, I created two Skeletons:
FACEMESH_TESSELATION_NO_IRISandFACEMESH_TESSELATIONto support the legacy code with optional refinement of the iris. However, there is a minor difference between the skeletons described in the mediapipe library and the ones in this PR. The skeletons represent the connections between each point on them, but there are 2 points out of the 478 that do not have a connection (the centre of the iris). Since theEdgeAnnotatorclass uses the SKELETONS_BY_VERTEX_COUNT variable for counting unique vertices in the skeleton, I connected each center to itself to have a unique vertex count of 478. This can be appreciate in the Google Colab notebook.The code was updated to look for the correct inference result using the
hasattrmethod, which makes easier to integrate theHandLandmarkerresults in the future (if it is planned).The docstring was updated to add a
FaceLandmarkerinference example. (The lines are probably too long but the pre-commit won't reformat)Docs