[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expo FaceDetector with inconsistent results on iOS in landscape mode #23475

Open
claitonlovato opened this issue Jul 11, 2023 · 3 comments
Open

Comments

@claitonlovato
Copy link

Minimal reproducible example

https://snack.expo.dev/@claitonlovato/reproducible-demo-for-face-detection-issues-in-landscape

Summary

When using Expo-Camera with Expo-FaceDetector, the results are very inconsistent when the app is in landscape mode and running on iOS. Both iPhones and iPads show inconsistent results, but the issue is more pronounced when testing on iPads.

The following is the logged messages of the reproducible sample in landscape (See "console.log..." on the faceDetect event):

LOG  No Face detected!
... 
LOG  1 Face detected all good!
LOG  No Face detected!
... Nothing for a while and same pattern repeats

Where simply rotating the device (same distance from face, etc) will result in the following logs:

LOG  1 Face detected all good!
LOG  1 Face detected all good!
LOG  1 Face detected all good!
LOG  1 Face detected all good!
...

Others have reported the same or similar issues recently with the issues getting closed without any actual resolution (e.g. #17250, 21875).
This is very important to us as users have mounted iPads in landscape mode to use the app and now this feature is useless for them.

Managed or bare workflow? If you have ios/ or android/ directories in your project, the answer is bare!
bare

What platform(s) does this occur on?
iOS

SDK Version (managed workflow only)
49.0.20

Reproducible demo
Bare app created with the react-native base template and Expo modules, CLI, etc added on
App.tsx:

import React from 'react';
import { useState, useRef, useEffect } from "react";

import {
  SafeAreaView,
  StatusBar,
  StyleSheet,
  Text,
  useColorScheme,
  View,
} from 'react-native';
import { Dimensions } from "react-native";
import { Camera, CameraType } from 'expo-camera';
import * as FaceDetector from "expo-face-detector";

const useScreenDimensions = () => {
  const [screenData, setScreenData] = useState(Dimensions.get("screen"));
  useEffect(() => {
    const subscription = Dimensions.addEventListener(
      'change',
      ({window, screen}) => {
        setScreenData(screen);
      },
    );
    return () => subscription?.remove();
  });

  return {
    ...screenData,
    isLandscape: screenData.width > screenData.height
  };
};

function App(): JSX.Element {
  const isDarkMode = useColorScheme() === 'dark';
  const [permission, requestPermission] = Camera.useCameraPermissions();
  const [scanMessage, setScanMessage] = useState("Looking for a face...");
  const screenData = useScreenDimensions();
  const camera = useRef();
  const [cameraReady, setCameraReady] = useState(false);
  const CameraReady = async() => {
    setCameraReady(true);
  };
  
  const active = !(
    !cameraReady
  );
  
  const faceDetect = ({ faces }) => {
    const numberOfFaces = faces.length;

    if (numberOfFaces === 0 ){
      console.log("No Face detected!");
      setScanMessage("No Face detected");
      return true;  
    }

    console.log(numberOfFaces + " Face detected all good!");
    setScanMessage("Great, found your face!");
    return true;
  };

  return (
      <SafeAreaView>
        <StatusBar
          barStyle={isDarkMode ? 'light-content' : 'dark-content'}
        />
        <View>
          <View
            style={{
              ...styles.container,
              flexDirection: screenData.isLandscape ? "row" : "column"
            }}
          >
            <View style={styles.statusContainer}>
              <Text
                style={{
                  ...styles.statusText
                }}
              >
                {scanMessage}
              </Text>
            </View>
            
            
            <Camera
              type={CameraType.front}
              style={{
                ...styles.camera,
                width: screenData.isLandscape
                  ? screenData.height
                  : screenData.width,
                height: screenData.isLandscape
                  ? screenData.height
                  : screenData.width

                // , marginTop: imagePadding, marginBottom: imagePadding
              }}
              ratio={"1:1"}
              ref={camera}
               => CameraReady()}
               message }) => console.log("Mounting Error: " + message)}
               message }) => console.log("Face Detection Error: " + message)}
               ? faceDetect : undefined}
              faceDetectorSettings={{
                mode: FaceDetector.FaceDetectorMode.fast,
                detectLandmarks: FaceDetector.FaceDetectorLandmarks.all,
                runClassifications: FaceDetector.FaceDetectorClassifications.all,
                minDetectionInterval: 100,
                tracking: false
              }}
            />
          </View>
        </View>
      </SafeAreaView>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: '#FFF',
    color: '#000'
  },
  statusText: {
    textAlign: "center",
    alignSelf: "center"
  },
  statusContainer: {
    justifyContent: "center",
    alignItems: "center",
    flexDirection: "column",
    flex: 1
  },
  camera: {}
});

export default App;

Environment

expo-env-info 1.0.5 environment info:
System:
OS: macOS 13.4.1
Shell: 5.9 - /bin/zsh
Binaries:
Node: 20.2.0 - /usr/local/bin/node
Yarn: 1.22.19 - /usr/local/bin/yarn
npm: 9.6.6 - /usr/local/bin/npm
Watchman: 2023.05.22.00 - /usr/local/bin/watchman
Managers:
CocoaPods: 1.12.1 - /usr/local/bin/pod
SDKs:
iOS SDK:
Platforms: DriverKit 22.4, iOS 16.4, macOS 13.3, tvOS 16.4, watchOS 9.4
IDEs:
Android Studio: 2022.2 AI-222.4459.24.2221.9971841
Xcode: 14.3.1/14E300c - /usr/bin/xcodebuild
npmPackages:
expo: ^48.0.0 => 48.0.20
react: 18.2.0 => 18.2.0
react-native: 0.71.8 => 0.71.8
Expo Workflow: bare

@claitonlovato claitonlovato added the needs validation Issue needs to be validated label Jul 11, 2023
@expo-bot expo-bot added needs review Issue is ready to be reviewed by a maintainer and removed needs validation Issue needs to be validated labels Jul 11, 2023
@expo-bot
Copy link
Collaborator

Thank you for filing this issue!
This comment acknowledges we believe this may be a bug and there’s enough information to investigate it.
However, we can’t promise any sort of timeline for resolution. We prioritize issues based on severity, breadth of impact, and alignment with our roadmap. If you’d like to help move it more quickly, you can continue to investigate it more deeply and/or you can open a pull request that fixes the cause.

@brentvatne brentvatne removed the needs review Issue is ready to be reviewed by a maintainer label Jul 12, 2023
@gwendall
Copy link
gwendall commented Aug 4, 2023

+1

@jrfurini
Copy link
jrfurini commented Aug 8, 2023

the same here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants