-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(firestore-bigquery-export): allow firestore database selection #1933
Conversation
@jauntybrain conflict |
…nvertase/support-multiple-databases
can you add this new param to the _emulator's env file, that might be the reason tests are failing |
Hi @jauntybrain / @pr-Mais : Can this feature for allowing firestore database selection be made available in the February release of the firestore-bigquery-export extension ? It would be really helpful for some of our projects where we are using multiple firestore databases in the same firebase project. |
Hi @jauntybrain / @pr-Mais - is there any way to speed up the release of this feature? |
@pr-Mais any chance we could get this merged and uploaded to https://extensions.dev? Given that support for multiple databases is now GA, this would be really helpful to have! |
Thanks all for your patience, we will release it ASAP. |
Hey all! Unfortunately, we have to pause the release of this feature for now. Extensions currently do not support v2 triggers that are required for named database selection (v1 trigger SDK does not support this feature). We will be monitoring Firebase updates and implement this feature as soon as the v2 function support is available. |
Then why does the Firestore SDK support setting the database for Cloud Functions v1? 🤔 Okay that option was added 6 years ago... |
@jauntybrain - some extensions, such as the Search Firestore with Algolia ( https://extensions.dev/extensions/algolia/firestore-algolia-search ) allow the Database ID to be configured, and we can specify Firestore database IDs other than the (default) database. How is it that some extensions have been able to do this without running into the v2 trigger limitations ? |
@jauntybrain well it seems like https://github.com/search?q=repo%3Aalgolia%2Ffirestore-algolia-search%20databaseId&type=code For the real-time index operation the database is not configured: |
@jauntybrain @pr-Mais couldn't the extension use EventArc triggers and use Firestore as an event source? 🤔 This would work with Cloud Functions V2 and seems to allow using non-default databases: |
Hi @IchordeDionysos, thank you for this suggestion. I'll review this approach to see if Eventarc triggers can be a suitable replacement for Firestore triggers until v2 support is added. |
Hi everyone. After extensive testing, it seems like this feature is working well after all. The Firestore trigger correctly recognizes doc changes from a secondary database, so no EventArc implementation is needed. We will be releasing this feature in v0.1.46, as originally intended. |
@jauntybrain - Thats great news ! |
@jauntybrain any rough idea when you'd be able to push the release? |
@jauntybrain, I'm curious what's the timeline for releasing the update containing this. Thanks! |
…1933) Co-authored-by: Mais Alheraki <mais.alheraki@gmail.com>
Hello @jauntybrain , we just tested the update but can't get it working. When installing the extension we get an error So it does not seem to work with v1 functions and that's also what's mentioned in the docs. |
Hi @alexregier, thanks for reporting this. Looking into it now. |
@alexregier I just tested version 0.1.48 and was able to successfully listen to changes in a secondary database instance with no errors. Could you please provide the full configuration of the extension with any sensitive information redacted? |
@jauntybrain I configured the extension via terraform, but this is the configuration to be used:
|
I'm seeing the same issue. Could it be because the named Firestore instance is not located in |
@neelance ours is also not in We have gotten it to work somehow, we didn't make any huge changes (only updated from version 0.1.46 to 0.1.48) I guess we also did the following:
|
@IchordeDionysos Are you sure that it actually worked? I also got a success message of the extension when I applied the config a second time, but the underlying Cloud Function was still in an error state and not working. |
@jauntybrain What is the status of this issue? I have just tested 0.1.48 and 0.1.49 and experience the same as @IchordeDionysos and @neelance I have also left a comment on this issue: #2072 (as I didn't see this issue earlier) |
Adds support for the multi-database feature recently introduced in Firestore.
Allows users to specify a database ID for their Firestore instance.
Includes non-breaking dependency and code changes for
firestore-bigquery-change-tracker
.Resolves #1743.