This issue is going to take some thought and likely be specific to each controller. i.e. identifying the specific errors that are not recoverable will be different for each cloud provider API call.

I believe we can solve this somewhat systemically if crossplane/crossplane#593 is merged. If requeue logic becomes a function of the error returned by CRUD operations then we can assert whether an error is recoverable and have the Reconciler scaffold avoid dequeueing accordingly, i.e.:

type recoverable struct{ error }

func (r recoverable) Recoverable() {}

func Recoverable(err error) error { return recoverable{err} }

func IsRecoverable(err error) bool {
	_, recoverable := err.(interface {
		Recoverable()
	})
	return recoverable
}

https://play.golang.org/p/H5eJBA8fi8y

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.