I absolutely love this and wanted it for some time, however be mindful that there are some limitations right now, especially for those of us in the EU:

• Aurora Serverless for MySQL only supports MySQL 5.6 right now, which I am not sure is still supported by Prisma 2 (the supported-databases.md only mentions MySQL 5.7). Aurora Serverless for MySQL 5.7 will likely launch this year, but it hasn't so far

• Aurora Serverless for PostgreSQL only supports PostgreSQL 10.7 right now, and is also only available in Ireland, not Frankfurt (Germany) in the EU so far
  No longer the case, see my comment below

• Data API for Aurora Serverless is only available in some regions so far, e.g. not in Frankfurt (Germany) (EU-Central-1), but in Ireland (EU-West-1)
  No longer the case, see my comment below

Linking my comment for relevance, as suggested: #2133 (comment)

Will love to do any testing when needed.

A few updates since my comment above.

• Data API for Aurora Serverless is now available in more regions, including eu-central-1 (EU/Frankfurt), see https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-rds-data-api-and-query-editor-available-additional-regions/

• Aurora Serverless for PostgreSQL (10.7) is now available in more regions, including eu-central-1 (EU/Frankfurt). It includes support for Data API for Aurora Serverless

May I have a bold idea here:

In terms of supporting the Data API, can we compile the whole GraphQL query into one SQL and execute it over Data API?

What a performance...

I'm using https://redwoodjs.com/ which deploys a graphql API as a serverless function. It would be great to have Data API support so that AWS Aurora Serverless can be used with such serverless applications.

More use cases described in this issue: #2133

I'm incredibly interested in the possibility of making this happen. Has anyone looked deeply at what it would take yet? I use Prisma all the time but have yet to actually go through the source code. If this is a manageable project it may be one I want to take on, but I'm wondering if anyone has looked into scope, or if Prisma has plans already in place for adding this?

There is Data API client which is also used by TypeORM https://github.com/ArsenyYankovsky/typeorm-aurora-data-api-driver

The Data API seems to be plagued with some issues related to casting types like json, dates etc which is handled by the above client jeremydaly/data-api-client#57

It seems like to be able to use this Prisma would have to explicitly provide the types to the Data API client.

@medv mentioned that they are using this with TypeORM and can probably share their experience.

We opted to use Jeremy Daly's data-api-client instead of the typeorm implementation (which I would not recommend, but this is my opinion). This approach has worked surprisingly well for our serverless use case. However, the dev ux has suffered a fair bit in comparison to what otherwise could have been with prisma.

@medv , would love a bit of expansion on why you ditched typeorm/typeorm-aurora-data-api-driver in favor of data-api-client.

TypeORM itself did not fit our team, so our decision was less about the data api implementation. data-api-client allowed us to retain our temporary ORM solution in the back-end while providing low latency queries and small bundle size through data-api-client for our serverless front-end served from lambda@edge (temporary because we are eagerly awaiting prisma with migrate and data-api to be well defined and widely used).

Is a data api driver something that is actively being worked on?

So, I asked a prisma developer this directly, and he took the question to the product team. I specified that I had a project launching in July, and if the integration would be done by summer, we would build with prisma. They said that currently they have no plans to prioritize this, and it would not likely be done this year. Obviously the more interest the more likely they are to prioritize, but right now I don’t think it is really on the radar.

Also would be interested in this

So, I asked a prisma developer this directly, and he took the question to the product team. I specified that I had a project launching in July, and if the integration would be done by summer, we would build with prisma. They said that currently they have no plans to prioritize this, and it would not likely be done this year. Obviously the more interest the more likely they are to prioritize, but right now I don’t think it is really on the radar.

That's extremely disappointing. @justinhandley what are you planning on doing?

can i use aurora serverless as prisma db or not? (or is this thread only about using data api for internal processing?)
https://www.prisma.io/docs/reference/database-reference/supported-databases

'cause aurora serverless is listed as supported db

@softmarshmallow, you can use aurora serverless. This conversation is about the Data API. In the fantasy of moving to a completely scalable and serverless environment, as well as having your database scalable, your endpoints would ideally also be scalable and served from edge servers. Unfortunately, with aurora serverless, you can’t access the database from outside of the region the database is in, so you can’t have an api served on edge lambdas connect. You could theoretically achieve this with the aurora data api. But, you are good to go.

@schickling himself opened this issue so I'm sure he sees the huge opportunity this would mean for developers using Prisma. In this whole JS world that is trending more and more towards Serverless, scalable database access is one of the largest bottlenecks. I believe Prisma would benefit hugely from this move and I wonder what the reasoning is to not offer support in the short and medium term.

Disclaimer: I do have to say though that I do not know how resource-intensive it would be to make these changes (time-wise). From an outsider perspective, wouldn't it "just" mean sending the SQL strings (that the engine already generates) via HTTP rather than a native connection? Looking forward to a quick response, if possible at all :)

On a separate topic but kind of related, I think it will be in the best interest of developers using Prisma to mention the current status of Data API and RDS Proxy compatibility in the docs and wherever else applicable. Right now this information is painfully hard to weed out. I have opened a PR for the same prisma/docs#1409

@medv - I'm wondering if you could share your experience using the Aurora Serverless Data API so far? I have talked to a number of people who had trouble using it at scale reporting issues such as large increase in response time, random failures and sometimes receiving wrong data. Have you experienced something like that or has it been smooth sailing?

@medv - I'm wondering if you could share your experience using the Aurora Serverless Data API so far? I have talked to a number of people who had trouble using it at scale reporting issues such as large increase in response time, random failures and sometimes receiving wrong data. Have you experienced something like that or has it been smooth sailing?

I can definitely confirm all that you said. I wonder how Aurora Serverless 2 would fare and looking at fauna in the meantime, but we have opted for a non-serverless Aurora with 1000 connection limit as a stopgap. Slight gritting teeth at the tape holding this all together. The idea is incredible from the getgo and in theory this is what we wish to do, but we have to bide our time with these interim limbo dances in the face of reality. Our usecase is not massive, but it's still about ~5mill users a month so plenty for us to invalidate a lot of approaches which we have so far.

1. Prisma on top of Data API would solve massive dev ux overhead, but may not entirely solve Aurora Serverless issues (may address some client layer issues over time due to sheer number of users and amazing prisma dev team having some ownership). This would already be the front-running solution in our eyes. Even with remaining issues, we would already bite the bullet and use this with workarounds on our end, whether it's cache fallbacks for slow or unavailable data, or retries etc. Perfectly happy to sustain the losses for the productivity boost and beautiful code approach.
2. Fauna with a prisma binding would be pretty ace, but probably has unforseen issues in of itself, I will be happy to report back after our experiments which are on the cards but have no real date at of now.
3. Ultimately, Prisma + whatever database driver that addresses connection pools would win the internet in our eyes after having played with a lot of arrangements.

Lambda@Edge deployments are absolutely incredible for whatever scale, and certainly help us enter a project with no growth expectation with our existing turnkey deployment while knowing full-well that it would be completely fine with sudden bursts of millions of people in the event of a business success. An hour to bootstrap a whole stack with scaling insurance puts us at absolute rest and allows us to extend massive cost savings to our partners or clients. The distributed nature is incredible and allows us to optimise on a per-region basis by replicating S3 and RDS resources with routing at edge, so it can grow into a super low latency solution when the need arises (getting there). This last hurdle is what keeps this from being truly great, and we're betting on prisma as we absolutely love the approach and the consistency of the prisma team's efforts.

Thank you @medv - that is extremely helpful!

One of our product teams is tasked with solving the problems you are describing. I would love to get on a call together with our pm to make sure we capture as much as possible. Would you be open to a call sometime in the coming weeks? You can contact me directly at schmidt@prisma.io

Do you know if this feature is included at least in the Prisma's roadmap?

Aurora Serverless v1 is reaching its End Of Life on Februray 28, 2023. And as long as I know Aurora Serverless v2 does not support data api client .

I don't think serverless v1 is reaching EoL

The thread link clarified that only "Amazon Aurora MySQL-Compatible Edition version 1" (which means MySQL 5.6) on Aurora Serverless v1 is EOL. "Aurora MySQL version 2" and "Aurora MySQL version 3" are still supported on "Aurora Serverless v1" and will be.

@nickygb I didn't realize v2 doesn't support Data API (which is definitely confirmed after deep-diving those threads, I guess I was inadvertently using v1 via SST). That being so, wouldn't Prisma be easier to implement for v2? It seems v2 uses traditional connection-handling, with connection-pooling via RDS Proxy and such. Has anyone tried just using Prisma as-is, connecting to an RDS Proxy -> Serverless v2, and see if it just works?

I'm trying to get to the bottom of v2's move from Data API, and I'm getting the sense they want to keep the good parts of serverless (scale up/down on ACU and storage) while reverting back the parts which caused compatibility issues, per this thread (even if that means traditionally-complex setups with VPC, Proxy, Nat, etc)

Prisma fully supports AWS Aurora Serverless (any version) as long as you use the traditional connection based connection string and not the Data API, see https://www.prisma.io/docs/reference/database-reference/supported-databases.

(RDS Proxy is another service with its own challenges: #5866 Some information: https://www.prisma.io/docs/guides/deployment/deployment-guides/caveats-when-deploying-to-aws-platforms#aws-rds-proxy)

Thanks for the reply. I really should have come to understand the difference between Data API & Aurora Serverless before going down this journey.

@lefnire

the reason we're sticking to v1 in SST is because it is the only viable way to use RDS and be VPC-less (thanks to data api).

that's a constraint we find important for building fully serverless systems for many reasons.

v2 isn't necessarily "better" - it's just choosing a different set of tradeoffs and we haven't found them compelling enough to move off of v1 - we're hoping it supports data api eventually

additionally Prisma's issues isn't directly with RDS - it's with Lambda. Prisma does not run well in Lambda due to its architecture

Prisma does not run well in Lambda due to its architecture

That's good to know, as I was just about to pros/cons an investigation on migrating my SST app to aurora v2. I'm already using a VPC for EFS in my SST app, so I'm fine with that (though I understand why you want to avoid that need for most users). I'll do some digging around the Lambda issues so I'm not pestering you here.

The main reason I was considering v2 is to use Prisma. I've been banging my head against kysely-data-api for a while. I'll chime in there if/when I can, but I just switched to raw SQL queries everywhere so I can launch fast. Even so I hit Data API snags, like no array support (IN (..), ARRAY[...], etc - aws/aws-sdk#9). This isn't the first such snag, so I'd rather take the VPC/Nat/Proxy hit than gotchas. But if the issue is more around Lambda usage.. anyway, I'll do some digging around. Thanks for the reply!

Prisma does not run well in Lambda due to its architecture

Is there any more information about this? Is this an official stance?

We are running Prisma on Lambda and haven't ran into any problems so far, but I'm really interested in reading more about this...

No, this is not an official stance and we do not agree with it. Prisma has its rough corners, especially in serverless environments, but those most often are not hard blockers and we are working on improving them.

If you want to discuss problems Prisma supposedly has with Lambda, please do so in a new discussion and not this feature request for AWS Data API support. Thanks.

@LinusU

Prisma requires large binaries to be shipped with your lambda function and started up on coldstarts. These add significant cold start time in lambda environments (upwards of 2 seconds) - this is well documented in several places. I find it kind of odd prisma is suggesting these are "supposed issues" and it is incredibly relevant to this issue because it's the entire reason why people want direct data api support

My primary motivation for using Data API is so that I can skip provisioning a VPC (slows down deployments, more complicated infra) and NAT gateways (money)

Of course if it can reduce cold starts in lambda that is critically important as well for us

AWS says Data API support in Aurora Serverless V2 is an internal feature request with no ETA, I haven't the foggiest notion what that signifies in relation to temporal concerns of the flesh

This issue contains multiple mentions of Prisma ORM not working well in Lambda - specifically by causing slow cold-starts.

This blogpost outlines work done over the past 9 months to improve Prismas startup performance, which benefits all users, but especially when deploying to serverless functions: https://www.prisma.io/blog/prisma-and-serverless-73hbgKnZ6t

Prismas startup performance is now on par with the best other ORMS, and faster than most of our competitors. Establishing a database connection still adds to the overall cold-start time, but Prisma adds no significant overhead.

This issue contains multiple mentions of Prisma ORM not working well in Lambda - specifically by causing slow cold-starts.

This blogpost outlines work done over the past 9 months to improve Prismas startup performance, which benefits all users, but especially when deploying to serverless functions: https://www.prisma.io/blog/prisma-and-serverless-73hbgKnZ6t

Prismas startup performance is now on par with the best other ORMS, and faster than most of our competitors. Establishing a database connection still adds to the overall cold-start time, but Prisma adds no significant overhead.

I can confirm this! I recently did a POC of TypeORM, MikrORM and Prisma in an AWS Lambdas via AWS CDK against Aurora Serverless V2 and the performance both with cold and warm starts was the same or better when using Prisma. The only remaining concern I have is with the large lambda size when using Prisma in comparison to the other ORMs which in theory should cause slower cold starts but I did not see that being an issue in my testing.

(We finally have to conclude our investigation and do a blog post, until then @raymondKelly: That matches our observation. As far as we can see, the problem is not "big lambda size" but "big lambda size because of many tiny files or lots of JavaScript that has to be parsed". Foruntately, our query engine only makes the Lamba bigger because of that one native code file, and hence usually does not impact cold start negatively.)

Hey, we have been working on something related for the past few weeks. We might have something to share and test soon.
If you are interested in helping us, please respond to this super short survey and leave your email address so we can get in touch with instructions: https://pris.ly/survey/driver-adapter-aws-data-api Thanks.

@janpio AWS just released RDS Data API support for Aurora serverless V2 https://aws.amazon.com/about-aws/whats-new/2023/12/amazon-aurora-postgresql-rds-data-api/ would it make sense that I open up another ticket for that?

Is there any update on the result of the survey linked and any update on if this adapter has made it onto the roadmap? I would think this is maybe the most popular serverless DB being used.

Great to see that AWS is moving here, although in pretty drastic ways.

AWS just released RDS Data API support for Aurora serverless V2 aws.amazon.com/about-aws/whats-new/2023/12/amazon-aurora-postgresql-rds-data-api would it make sense that I open up another ticket for that?

I think with AWS retiring Aurora Serverless v1, and the Data API connected to that, this issue automatically becomes a feature request for adding support for the Data API based on the now only available option with Aurora Serverless v2 - so no issue necessary really.

Is there any update on the result of the survey linked and any update on if this adapter has made it onto the roadmap? I would think this is maybe the most popular serverless DB being used.

The survey got a small number of responses for AWS Data API, <4%. PlanetScale and Neon are much more popular in our userbase (>20% of responses each). So although I recognize that some really want to use AWS Data API, this does not put it near the top of our priorities unfortunately.

Driver adapters of course can be written by anyone, so maybe someone from the community wants to pick this up and figure out if this is possible.

@janpio That is Understandable, thank you for your reply. My only feedback would be that Aurora Serverless on AWS will likely have a much larger user base in the future than the others mentioned and I would think that Prisma would be strategically best positioned in aligning support for that userbase so as to grow your own userbase. However, I understand I don't know the ins and outs of the Prisma userbase or strategic focus going forward so it's speculative on my end.

@janpio Is there any good tutorial or documentation out there about writing Prisma driver adapters? Thanks!

No, they are super fresh and no documentation yet. Looking at our codebase (https://github.com/prisma/prisma/tree/main/packages, adapter-*) should give you an easy understanding though. They are basically a defined API to be exported, and then just the internal are different for each driver adapter - calling the underlying database client or driver. If you have more questions, maybe start a discussion in this repo and I will try to be helpful (feel free to ping me when creating it).

No, they are super fresh and no documentation yet. Looking at our codebase (https://github.com/prisma/prisma/tree/main/packages, adapter-*) should give you an easy understanding though. They are basically a defined API to be exported, and then just the internal are different for each driver adapter - calling the underlying database client or driver. If you have more questions, maybe start a discussion in this repo and I will try to be helpful (feel free to ping me when creating it).

This is a helpful starting point, thanks!!

@mariomnts Pls let me know if you decide to work on an adapter as I may be interested in helping.

@mariomnts Pls let me know if you decide to work on an adapter as I may be interested in helping.

No, I haven't looked for the time, it's a stressful time of the year to me. I can do as @janpio suggested and create a discussion as I hace some questions regarding the internals of the adapters though

Discussion created: #22721

Would still love to see this supported, and would give a modest sponsorship to anyone who works on a AWS Data API driver. I've been told to port to Drizzle but I just can't seem to get behind their api.