[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying failure policies #482

Merged
merged 56 commits into from
Nov 27, 2019
Merged

Conversation

merlinnot
Copy link
Contributor
@merlinnot merlinnot commented Jun 21, 2019

This PR:

  • Makes runtime configuration validation more precise.
  • Adds a FailurePolicy, which can be specified for all functions.
  • Adds a warning message when FailurePolicy is used for https functions. Originally I planned to express it using more correct typings, it's doable, but not very clean. It's tricky because the runtime options are specified before the function trigger is chosen, so specifying a retry policy would result in limiting the number of options for triggers (removing https). We'd either need two classes returned conditionally based on the config (EventTriggeredFunctionBuilder when retry policy is specified, current FunctionBuilder otherwise) or we can just have this warning and ignore this option for https functions. Let me know which approach do you like better.
  • Exposes FailurePolicy in __trigger, so it can be consumed by firebase-tools.
  • Adds tests for setting and exposing failurePolicy.

I'm wondering how to approach releasing this functionality. Should we first implement support for it in firebase-tools, release a new version of it and start requiring it in peerDependencies? It would be weird if it's allowed to specify it here, but the setting is ignored by the CLI.

Description

Resolves #425.

Code sample

The API change is described in the issue linked above.

@merlinnot merlinnot changed the title Define an interface for FailurePolicy Allow specifying failure policies Jun 21, 2019
@merlinnot merlinnot marked this pull request as ready for review June 24, 2019 11:27
@merlinnot
Copy link
Contributor Author

Travis fails on Node.js 10, because it can't install it. CI passes on Node.js 8.

@thechenky
Copy link
Contributor

Hey @merlinnot just rerun the job if it fails on stuff like installing Node 10, it usually is a transient error. I just reran your last one and it passed.

@merlinnot
Copy link
Contributor Author

Great, thanks! The PR is ready for a first pass, please not the question in the description of the PR.

@merlinnot
Copy link
Contributor Author

@thechenky Hey, just a friendly ping for a review :)

@thechenky
Copy link
Contributor

Ah, will get to this in a bit - for future reference, please add me as reviewer at the top so that I understand it's ready for review.

Copy link
Contributor
@thechenky thechenky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a loooooooooot of code changes here that are unrelated to the actual feature which makes it very hard to review. I've made a lot of comments here, mostly around pulling these changes out into separate PRs. I'm thinking you need two extra PRs - one for formatting/comment changes, and one for refactors you made around pulling out configuration and changing opts => options.

Please do this, add me as reviewer on all the other PRs so that I can review them separately. Then I will review the code that's left in this PR, which should just be the feature. Also btw, we will be needing CLI changes to parse __trigger to pluck out the extra retry options and supply them in the API to GCF.

changelog.txt Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
integration_test/functions/src/database-tests.ts Outdated Show resolved Hide resolved
integration_test/functions/src/firestore-tests.ts Outdated Show resolved Hide resolved
src/providers/https.ts Outdated Show resolved Hide resolved
src/providers/pubsub.ts Outdated Show resolved Hide resolved
src/providers/remoteConfig.ts Outdated Show resolved Hide resolved
src/providers/storage.ts Outdated Show resolved Hide resolved
src/cloud-functions.ts Outdated Show resolved Hide resolved
@merlinnot
Copy link
Contributor Author

I'll extract opts -> options rename and comments fixes to separate PRs once the three new ones I created are merged, it will be easier for me to extract these changes.

@thechenky
Copy link
Contributor

Sounds good. Reviewing the others now.

@samtstern
Copy link
Contributor

@merlinnot looks like you might need to run prettier on this one more time before we can merge? I merged master into your branch so please pull (or just force-push your own thing)

@merlinnot
Copy link
Contributor Author

@samtstern Done!

@kevinajian kevinajian merged commit 8039e02 into firebase:master Nov 27, 2019
@kevinajian
Copy link
Contributor

Thanks @merlinnot !

@merlinnot merlinnot deleted the fix-issue-425 branch November 27, 2019 21:51
Copy link
Contributor
@joehan joehan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tinaliang added a commit that referenced this pull request Feb 5, 2020
tinaliang added a commit that referenced this pull request Feb 5, 2020
* Revert "Allow specifying failure policies (#482)
@rhodgkins
Copy link
Contributor

Any ideas when this is going to be re-merged back in / released?

@tinaliang
Copy link
Contributor

Good call! @samtstern Is the work in the CLI complete?

@samtstern
Copy link
Contributor

I know this sounds pathetic but no matter what I tried I could not get this to work, the GCF API did not seem to be taking my PATCH requests.

@tinaliang I'll bring you into the internal thread.

@andersonaddo
Copy link

giphy
Just found this PR, I'm happy this is in the works!
Just hoping it'll be merged again soon :)

@niklv
Copy link
niklv commented Aug 13, 2020

Hi

I find workaround, you can define your functions like this:

const functions = require('firebase-functions')

const fn = functions.firestore
  .document('/collection/{item_id}')
  .onWrite(async (change, context) => {
    // duno
    return 0
  })

module.exports = new Proxy(fn, {
  get(target, phrase) {
    const obj = target[phrase]
    if (phrase === '__trigger') obj.eventTrigger.failurePolicy = { retry: {} }
    return obj
  }
})

@samtstern samtstern mentioned this pull request Aug 13, 2020
@cjimmy
Copy link
cjimmy commented Oct 23, 2020

@merlinnot I may be mistaken here, but the current firebase docs do not reflect this change, correct? https://firebase.google.com/docs/reference/functions/function_builder_.functionbuilder#runwith

@samtstern
Copy link
Contributor

@cjimmy you're right looks like we need to regenerate the reference docs, thanks for the reminder!

@cjimmy
Copy link
cjimmy commented Nov 3, 2020

Here are some notes, for my future self and others.

For anyone looking add retries to firebase functions, without having to use GCP console, this PR is it and enabled in firebase-functions 3.10.0. Since it's not entirely obvious, here's a quick example of the syntax:

export const myFirebaseFunc = functions
  .runWith({
    failurePolicy: {
      retry: {},
    },
    memory: '512MB',
    timeoutSeconds: 60,
  })
  .firestore.document('/path/to/some/doc')
  .onCreate(async (snap, context) => {
   /* do stuff */
 })

Right now, it looks like the failure policy is simply on or off. Thus, this is equivalent

export const myFirebaseFunc = functions
  .runWith({
    failurePolicy: true,
    memory: '512MB',
    timeoutSeconds: 60,
  })
  .firestore.document('/path/to/some/doc')
  .onCreate(async (snap, context) => {
   /* do stuff */
 })

Some caveats:

  • You'll also have to deploy with --force
  • You can enable retries only on triggered functions, not http-called functions.
  • It would be idiotic not to build in some safeguards. Retry policy maxes out at 7 days, and bills just like any other function invocation, so if you have some unhandled error, it could repeatedly run for a full week. You can use context.eventTimestamp to know when the first attempt roughly started.

Read this: https://firebase.google.com/docs/functions/retries and make sure your function in idempotent.

It was also difficult to discover what to return to force a retry or avoid a retry.
Triggered Firebase functions must return a Promise.

A Firebase function will retry if:

  • it returns a rejected promise
  • throws an exception
  • or the function times out

That means that if you run into an error that you know won't eventually resolve itself with a retry (i.e. you want to stop the function execution and not retry), you can return Promise.resolve({message: 'some message'});

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Intent to implement: retry option for event triggered functions