I can‘t Figure Out How to Get My Earth Entity to Rotate on its Axis. This is a follow up post from a previous Apple Developer forum post. How would I have the earth (parent) entity rotate CCW underneath the orbiting starship child? I tried adding the following code block to the RealityView but it is not working: if let rotatingEarth = starshipEntity.findEntity(named: "Earth") { rotatingEarth.transform.rotation = simd_quatf.init(angle: 360, axis: SIMD3(x: 0, y: 1, z: 0)) if let animation = try? AnimationResource.generate(with: rotatingEarth as! AnimationDefinition) { rotatingEarth.playAnimation(animation) } } Any advice on getting the earth to rotate? I tried reviewing the Hello World WWDC23 project code, but I was unable to understand the complexity and how that sample project got the earth to rotate. i want to do this for visionOS 1.2. I realize there are some new animation and possible other capabilities coming up in vision 2.0 but I want to try to address this issue in the current released visionOS version.

Hey, is there a way to create a good ground shadow shader? I'm using a ground with an unlit material and I can't get the ground shadow to work properly. If I use a PBR texture it works better, but i can barely see it and I want to control the intensity more.

I'm currently streaming synchronised video and depth data from my iPhone 13, using AVFoundation, video set to AVCaptureSession.Preset.vga640x480. When looking at the corresponding images (with depth values mapped to a grey colour map), (both map and image are of size 640x480) it appears the two feeds have different fields of view, with the depth feed zoomed in and angled upwards, and the colour feed more zoomed out. I've looked at the intrinsics from both the depth map, and my colour sample buffer, they are identical. Does anyone know why this might be? My setup code is below (shortened): import AVFoundation import CoreVideo class VideoCaptureManager { private enum SessionSetupResult { case success case notAuthorized case configurationFailed } private enum ConfigurationError: Error { case cannotAddInput case cannotAddOutput case defaultDeviceNotExist } private let videoDeviceDiscoverySession = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInTrueDepthCamera], mediaType: .video, position: .front) private let session = AVCaptureSession() public let videoOutput = AVCaptureVideoDataOutput() public let depthDataOutput = AVCaptureDepthDataOutput() private var outputSynchronizer: AVCaptureDataOutputSynchronizer? private var videoDeviceInput: AVCaptureDeviceInput! private let sessionQueue = DispatchQueue(label: "session.queue") private let videoOutputQueue = DispatchQueue(label: "video.output.queue") private var setupResult: SessionSetupResult = .success init() { sessionQueue.async { self.requestCameraAuthorizationIfNeeded() } sessionQueue.async { self.configureSession() } sessionQueue.async { self.startSessionIfPossible() } } private func requestCameraAuthorizationIfNeeded() { switch AVCaptureDevice.authorizationStatus(for: .video) { case .authorized: break case .notDetermined: AVCaptureSession sessionQueue.suspend() AVCaptureDevice.requestAccess(for: .video, completionHandler: { granted in if !granted { self.setupResult = .notAuthorized } self.sessionQueue.resume() }) default: setupResult = .notAuthorized } } private func configureSession() { if setupResult != .success { return } let defaultVideoDevice: AVCaptureDevice? = videoDeviceDiscoverySession.devices.first guard let videoDevice = defaultVideoDevice else { print("Could not find any video device") setupResult = .configurationFailed return } do { videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice) } catch { setupResult = .configurationFailed return } session.beginConfiguration() session.sessionPreset = AVCaptureSession.Preset.vga640x480 guard session.canAddInput(videoDeviceInput) else { print("Could not add video device input to the session") setupResult = .configurationFailed session.commitConfiguration() return } session.addInput(videoDeviceInput) if session.canAddOutput(videoOutput) { session.addOutput(videoOutput) if let connection = videoOutput.connection(with: .video) { connection.isCameraIntrinsicMatrixDeliveryEnabled = true } else { print("Cannot setup camera intrinsics") } videoOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)] } else { print("Could not add video data output to the session") setupResult = .configurationFailed session.commitConfiguration() return } if session.canAddOutput(depthDataOutput) { session.addOutput(depthDataOutput) depthDataOutput.isFilteringEnabled = false if let connection = depthDataOutput.connection(with: .depthData) { connection.isEnabled = true } else { print("No AVCaptureConnection") } } else { print("Could not add depth data output to the session") setupResult = .configurationFailed session.commitConfiguration() return } let depthFormats = videoDevice.activeFormat.supportedDepthDataFormats let filtered = depthFormats.filter({ CMFormatDescriptionGetMediaSubType($0.formatDescription) == kCVPixelFormatType_DepthFloat16 }) let selectedFormat = filtered.max(by: { first, second in CMVideoFormatDescriptionGetDimensions(first.formatDescription).width < CMVideoFormatDescriptionGetDimensions(second.formatDescription).width }) do { try videoDevice.lockForConfiguration() videoDevice.activeDepthDataFormat = selectedFormat videoDevice.unlockForConfiguration() } catch { print("Could not lock device for configuration: \(error)") setupResult = .configurationFailed session.commitConfiguration() return } session.commitConfiguration() } private func addVideoDeviceInputToSession() throws { do { var defaultVideoDevice: AVCaptureDevice? defaultVideoDevice = AVCaptureDevice.default( .builtInTrueDepthCamera, for: .depthData, position: .front ) guard let videoDevice = defaultVideoDevice else { print("Default video device is unavailable.") setupResult = .configurationFailed session.commitConfiguration() throw ConfigurationError.defaultDeviceNotExist } let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice) if session.canAddInput(videoDeviceInput) { session.addInput(videoDeviceInput) } else { setupResult = .configurationFailed session.commitConfiguration() throw ConfigurationError.cannotAddInput } }

Hi All, I am trying to build a new iOS app by following https://developer.apple.com/videos/play/wwdc2024/10163/?time=67 When I trying to remove all legacy VN I am getting error, I would appreciate if someone can help me get up to speed with the new Vision API

We have a lot of resources made using 3Dmaxs design software. How can the USDZ exported from 3dmax be used in the vision project. The preview in the folder on the Apple computer is correct. However, it is not possible to parse the texture in the xcode project. The difference in USDZ data compared to Blender is that the texture data exported by 3dmax has an additional node graph node. Is there any export plugin support?

I try to use the new VNCalculateImageAestheticsScoresRequest API. Code is compiling and running but delivers the same result for every image Xcode 16 Beta 2 Simulator Did I missing anything ?

"On the latest iOS 18 beta 2, the OCR API，the Translate App and Live Text performs very poorly in recognizing Japanese."

Vision pro cannot capture recordings over 16kHz to 24kHz at a sampling rate of 48kHz, why? Or can you tell me how to configure it?Vision pro cannot capture recordings over 16kHz to 24kHz at a sampling rate of 48kHz, why? Or can you tell me how to configure it?

I'm seeking insight on why the new VisionOS Barcode Scanning API is categorized as an Enterprise API and restricted only for proprietary and in-house apps. I understand Apple's focus on privacy and I can see how this restriction could make sense for other Enterprise APIs like main camera access and passthrough screen capture. Why is barcode scanning restricted from open apps? What makes barcode scanning more of a risk to privacy versus the unrestricted APIs for object tracking, image tracking, or hand tracking?

Hi everyone, I'm curious about the capabilities of ARKit's object tracking feature. Specifically, I'd like to know: Is there a size limit for the objects that can be tracked? Can ARKit differentiate between two objects with the same shape but different models (e.g., different colors)? Are objects with single colors and generic shapes (like squares or circles) effectively trackable? Any insights or examples from your experiences would be greatly appreciated! Thanks in advance.

I'm looking for a solution to take a picture or point the camera at a piece of clothing and match that image with an image the user has stored in my app. I'm storing the data in a Core Data database as a Binary Data object. Since the user also takes the pictures they store in the database I think I cannot use pre-trained Core ML models. I would like the matching to be done on device if possible instead of going to an external service. That will probably describe the item based on what the AI sees, but then I cannot match the item with the stored images in the app. Does anyone know if this is possible with frameworks as Vision or VisionKit?

Hello, I want to capture video from Vision Pro in the Vision OS app. I am referring to the (https://developer.apple.com/videos/play/wwdc2024/10139/) Apple video and their code. step like below import ARKit com.apple.developer.arkit.main-camera-access.allow = true in info.plist Do below code func loadCameraFeed() async { // Main Camera Feed Access Example let formats = CameraVideoFormat.supportedVideoFormats(for: .main, cameraPositions:[.left]) let cameraFrameProvider = CameraFrameProvider() var arKitSession = ARKitSession() var pixelBuffer: CVPixelBuffer? await arKitSession.queryAuthorization(for: [.cameraAccess]) do { try await arKitSession.run([cameraFrameProvider]) } catch { return } guard let cameraFrameUpdates = cameraFrameProvider.cameraFrameUpdates(for: formats[0]) else { return } print(cameraFrameUpdates) for await cameraFrame in cameraFrameUpdates { print(cameraFrame) guard let mainCameraSample = cameraFrame.sample(for: .left) else { continue } pixelBuffer = mainCameraSample.pixelBuffer } } I want to convert "pixelBuffer" into video streaming and show it in a frame like iOS. Please guide me on how to achieve my next step. I am blank after this code.

I have created an archive for both iOS and MacOS versions of my app by doing the the following steps Destinations select Build Any Mac (Mac Catalyst, arm64, x86_64) Product > Archive However when doing the same steps for VisionOS I get an error Invalid Run Destination I have selected both destinations, visionOS Simulator and Build any VisionOS simulator device (arm64, x86_64) I am able to run the app and test, now I would like to upload to AppStoreConnect for TestFlight and App Store submission.

I have created a Hand Action Classification project following the guidelines which causes the Create ML tool to provide the very cryptic "Unexpected Error". The feature extraction phase is fine, with the error occurring during model training after the tool reports completion of the first five training iterations as it moves on to report the next ten. The project is small with 3 training classes and 346 items I have tried to vary the frame rate and action duration, with all augmentations unset, but the error still persists. Can you please confirm how I may get further error diagnostic information so that I can determine why Create ML is unable to work with my training data? Mac OS is Sonoma 14.5 on an iMac 24-inch, M1, 2021. Create ML is Version 5.0 (121.4)

HI eveyone i've read that USDZ supports LOD to have 3 meshes with high medium and low polygon detail to be visible depending on the distance from the user to the entity... but i dont know how to use it... any experience or... god bless you... a downloadable file with a sample??? thanks a lot !!!!

this week i was watching https://developer.apple.com/videos/play/wwdc2024/10105/ with the amazing "configuration" feature to change the color or mesh straight in quick look, but i tried a lot with goarounds but nothing bring me to success how do i write in the usda files? anytiome i overwrite the usda even with just a "{}" inside... Reality composer pro rejects the file to be open again where is the developer man in the tutorial writing the usda? how is the usda compressed in usdz? (none of the compressors i tried accepeted the modified usda file) this is the code it's suggested in the video #usda 1.0 ( defaultPrim = "iPhone" ) def Xform "iPhone" ( variants = { string Color = "Black_Titanium" } prepend variantSets = ["Color"] ) { variantSet "Color" = { "Black_Titanium" { } "Blue_Titanium" { } "Natural_Titanium" { } "White_Titanium" { } } } but i dont understand how to do it with my own files,

I'm playing with the new Vision API for iOS18, specifically with the new CalculateImageAestheticsScoresRequest API. When I try to perform the image observation request I get this error: internalError("Error Domain=NSOSStatusErrorDomain Code=-1 \"Failed to create espresso context.\" UserInfo={NSLocalizedDescription=Failed to create espresso context.}") The code is pretty straightforward: if let image = image { let request = CalculateImageAestheticsScoresRequest() Task { do { let cgImg = image.cgImage! let observations = try await request.perform(on: cgImg) let description = observations.description let score = observations.overallScore print(description) print(score) } catch { print(error) } } } I'm running it on a M2 using the simulator. Is it a bug? What's wrong?

The DINO v1/v2 models are particularly interesting to me as they produce embeddings for the detected objects rather than ordinary classification indexes. That makes them so much more useful than the CNN based models. I would like to prepare some of the models posted on Huggingface to run on Apple Silicon, but it seems that the default conversion with TorchScript will not work. The other default conversions I've looked at so far also don't work. Conversion based on an example input doesn't capture enough of the model. I know that some have managed to convert it as I have a demo with a coreml model that seems to work, but I would like to know how to do the conversion myself. Has anyone managed to convert any of the DINOv2 models?

In my app, I'm setting the app language to Japanese. On iPhone and iPad, the Apple sign up button still displays correctly in Japanese: "Appleでサインイン". However, when running on vision Pro, the sign up button displays in English with the text "Sign up with Apple". Is this a vision pro error? The code setting the app language is Japanese: `CFBundleDevelopmentRegion ja_JP Photos displayed on iphone and vision pro. On Iphone: On vision Pro code signup button apple: (signInAppleButton as UIControl).cornerRadius = 2.0 let tapGesture = UITapGestureRecognizer(target: self, action: #selector(handleAuthorizationAppleIDButtonPress)) signInAppleButton.addGestureRecognizer(tapGesture) signInAppleStackView.insertArrangedSubview(signInAppleButton, at: 0) Hope you can help me why Vision Pro displays English text?

The new Mac virtual display feature on visionOS 2 offers a curved/panoramic window. I was wondering if this is simply a property that can be applied to a window, or if it involves an immersive mode or SceneKit/RealityKit?