[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Unhandled error cleaning up build images." - deploying from same GCP project #4757

Open
akauppi opened this issue Jul 20, 2022 · 5 comments
Open

"Unhandled error cleaning up build images." - deploying from same GCP project #4757

akauppi opened this issue Jul 20, 2022 · 5 comments
Assignees
@taeold
Labels
type: bug

Comments

@akauppi
Copy link
akauppi commented Jul 20, 2022 •
edited
Loading

[REQUIRED] Environment info

  • firebase-tools: 11.3.0
  • Platform: Cloud Build; same GCP project as the Firebase project

[REQUIRED] Test case

  • deploy a Firebase project with Cloud Functions, "onto itself", i.e. using the same GCP project that Firebase has automatically created.

[REQUIRED] Steps to reproduce

skip

[REQUIRED] Expected behavior

Deployment works, since the project has enough access rights.

Firebase would clean up Cloud Function deployment cache, as it aims to do.

[REQUIRED] Actual behavior

Deployment fails.

The cleanup fails.

Here's an excerpt from the log:

Step #6: === Deploying to 'groundlevel-jun-22'...
Step #6: 
Step #6: i  deploying firestore, functions
...
Step #6: i  functions: ensuring required API cloudfunctions.googleapis.com is enabled...
Step #6: i  functions: ensuring required API cloudbuild.googleapis.com is enabled...
Step #6: i  artifactregistry: ensuring required API artifactregistry.googleapis.com is enabled...
Step #6: ✔  artifactregistry: required API artifactregistry.googleapis.com is enabled
Step #6: ✔  functions: required API cloudbuild.googleapis.com is enabled
Step #6: ✔  functions: required API cloudfunctions.googleapis.com is enabled
Step #6: i  functions: preparing codebase default for deployment
Step #6: i  functions: preparing ./functions directory for uploading...
Step #6: i  functions: packaged /workspace/packages/backend/functions (53.92 KB) for uploading
Step #6: i  firestore: latest version of ./firestore.rules already up to date, skipping upload...
Step #6: ✔  firestore: deployed indexes in ./firestore.indexes.json successfully
Step #6: ✔  functions: ./functions folder uploaded successfully
Step #6: ✔  firestore: released rules ./firestore.rules to cloud.firestore
Step #6: i  functions: updating Node.js 16 function userInfoShadow_2(europe-west2)...
Step #6: ⚠  functions: failed to update function projects/groundlevel-jun-22/locations/europe-west2/functions/userInfoShadow_2
Step #6: Failed to update function projects/groundlevel-jun-22/locations/europe-west2/functions/userInfoShadow_2
Step #6: 
Step #6: Functions deploy had errors with the following functions:
Step #6: 	userInfoShadow_2(europe-west2)
Step #6: i  functions: cleaning up build files...
Step #6: ⚠  functions: Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at https://console.cloud.google.com/artifacts/docker/groundlevel-jun-22/europe-west2/gcf-artifacts
Step #6: 
Step #6: Error: There was an error deploying functions
Finished Step #6
ERROR
ERROR: build step 6 "us-central1-docker.pkg.dev/ci-builder/builders/firebase-emulators:11.3.0" failed: step exited with non-zero status: 2

The way I read that is that firebase-tools first checks that there are enough access rights:

Step #6: i  artifactregistry: ensuring required API artifactregistry.googleapis.com is enabled...
Step #6: ✔  artifactregistry: required API artifactregistry.googleapis.com is enabled

..but it then tries to do something that's beyond those rights (delete the earlier cached images).

image

Screenshot of the cached images

Attempting work-around

I removed the cached images manually, and retried the deployment.

It still fails.

I removed the user_info_shadow2__ folder, leaving an empty gcf-artifacts. Retried.

It still fails in the same way.

Since there were some artifacts there from earlier runs (7 and 12 days ago), either this is something in 11.3.0 (I upgraded in the mean time), or I may have inadvertently changed some access rights.

More info

Manual deployment works, with firebase-tools 11.3.0. That is, my access rights (as the owner of the GCP project) are enough.

This also used to work, like months back, but I was using Container Registry at the time. I have not done CI deployments since moving to Artifact Registry.

image

Screenshot of the gcd-artifacts folder

Note that the deployment region is europe-west2, a bit unusual.
@akauppi akauppi added the bug label Jul 20, 2022
@akauppi akauppi changed the title "Unhandled error cleaning up build images." - what rights should I give to the GCP project? "Unhandled error cleaning up build images." - deploying from said GCP project Jul 20, 2022
@akauppi akauppi changed the title "Unhandled error cleaning up build images." - deploying from said GCP project "Unhandled error cleaning up build images." - deploying from same GCP project Jul 20, 2022
@akauppi akauppi mentioned this issue Jul 20, 2022
Closed
@akauppi
Copy link
Author
akauppi commented Jul 21, 2022 •
edited
Loading

Corrections:

It doesn't have to do with firebase-tools version.

I needed to enable Service Account User:

image

Since this is not mentioned in the Firebase docs (I believe), I'll leave this issue open. Let me know if you wish me to edit it.

Note: The inability to clean up was separate from the actual error that fails the deployment. Can do two tickets if you think both of these are worth doing something about.

khenneuse added a commit to khenneuse/cloud-builders-community that referenced this issue Nov 17, 2022
@khenneuse
Update setup documentation for Firebase build image
53e3966 
While trying to determine why cloud function that I was working with in
the build were not being deployed I found an open issue.
firebase/firebase-tools#4757

This change is to update the documentation to reflect the need for
not just the Firebase Admin role in the builds, but also the
Service Accounts role.

Thanks to @akauppi for discovering the issue. It was the same issue
I had spent multiple hours trying to figure out.
@khenneuse khenneuse mentioned this issue Nov 17, 2022
Open
@khenneuse
Copy link

Thanks @akauppi I created a pull request to update the documentation in the community build repo

There should be an update to the firebase tool to print a warning about the service account not having access when that happens. It would be a clearer signal as to the issue.

The build did pass when run as a single step

  - name: gcr.io/<PROJECT_ID>/firebase
  args: ["deploy", "--project=<PROJECT_ID>"]

But failed when run as individual steps

  - name: gcr.io/<PROJECT_ID>/firebase
    args:
      ["deploy", "--project=<PROJECT_ID>", "--", "--force"]
  - name: gcr.io/<PROJECT_ID>/firebase
    args: ["deploy", "--project=<PROJECT_ID>", "--"]
  - name: gcr.io/<PROJECT_ID>/firebase
    args: ["deploy", "--project=<PROJECT_ID>", "--"]
  - name: gcr.io/<PROJECT_ID>/firebase
    args: ["deploy", "--project=<PROJECT_ID>", "--"]

This may be an issue inside of the package that it is not capture or checking the exit code of each step. Therefore only the exit code of the last step in the chain is reported to Cloud Build.

@zouhirdev
Copy link
zouhirdev commented Mar 31, 2024 •
edited
Loading

@akauppi
hello how can i fix this please im new to firebase im having the same error
i cant find the service account permissions to edit them please help

@joehan
Copy link
Contributor
joehan commented Apr 2, 2024

Hey @zouhirdev - you should be able to see and edit these permissions here: https://console.cloud.google.com/cloud-build/settings/service-account

If you are unable to see that page, you might not have the necessary permissions on your project, and should ask a project owner to help you out.

@spkrak
Copy link
spkrak commented Jun 19, 2024 •
edited
Loading

Hello, I've executed configuration steps mentioned by @joehan, yet I'm still receiving same warning after deployment (Unhandled error cleaning up build images. This could result in a small monthly bill if not corrected. You can attempt to delete these images by redeploying or you can delete them manually at...) and docker image is not deleted from artifact repository from the project so I need to remove it manually every time. Any suggestions on how to proceed further?

@busanTeddyBear busanTeddyBear mentioned this issue Jun 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@taeold taeold

Labels
type: bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@akauppi @joehan @khenneuse @taeold @spkrak @zouhirdev @christhompsongoogle