-
-
Notifications
You must be signed in to change notification settings - Fork 371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Builtin TestCase from python 3.8 #416
Comments
Hi, yes I noticed the built in async test support in py3.8, but decided to leave our hack of asynctest due to still having to support 3.7 At some point we probably need to rework the test tools. You're welcome to start 😄 |
Will do |
I had a try on tidying up the test cases using unittest but I have to admit, it is not an easy task, So first and foremost Another thing that I noticed is some methods that are the same as So before continuing I want to raise a couple of points with you @grigi and any other contributor that is interested in this matter, so that I do not waste time introducing a change that might not be accepted.
Last but not least, would you be open to move some testing code outside of the |
Yes, I did mention that this may be the worst code in the entire repo 😭
Yes, definitely. We would need to have some kind of support for existing customers. Possibly keeping 0.16 alive for a few months to allow migration.
The current I think the goal is to reduce the amount of boilerplate the user would need for testing, and if simple inheritance can do that for us, then I'd say go for it!
Sure. 🤔 That could be part of the solution to allow people to migrate to the new testrunner, we can leave the old, messy one in Where do you want to put it? |
We should look at supporting the minimal work at migrating the test DB between event loops, even if we only use it for some variants of test classes (or pytest support). |
Okay then I will have a try at creating a TestCase in |
|
This is much more complicated than I thought, I am having issues with the implementation of
Apart from that, this is what I came up with, some notable changes are the adding of db_url, For the import inspect
from typing import List
from unittest.async_case import IsolatedAsyncioTestCase
from tortoise import Tortoise, current_transaction_map, generate_config
SimpleTestCase = IsolatedAsyncioTestCase
class IsolatedTestCase(IsolatedAsyncioTestCase):
"""
An asyncio capable test class that will ensure that an isolated test db
is available for each test.
Use this if your test needs perfect isolation.
Note to use ``{}`` as a string-replacement parameter, for your DB_URL.
That will create a randomised database name.
It will create and destroy a new DB instance for every test.
This is obviously slow, but guarantees a fresh DB.
If you define a ``tortoise_test_modules`` list, it overrides the DB setup module for the tests.
"""
tortoise_test_modules: List[str] = []
db_url: str = ''
async def asyncSetUp(self) -> None:
config = generate_config(self.db_url, app_modules={'models': self.tortoise_test_modules},
testing=True, connection_label='models')
await Tortoise.init(config, _create_db=True)
await Tortoise.generate_schemas(safe=False)
self._connections = Tortoise._connections.copy()
async def asyncTearDown(self) -> None:
Tortoise._connections = self._connections.copy()
await Tortoise._drop_databases()
class TruncationTestCase(IsolatedTestCase):
"""
An asyncio capable test class that will truncate the tables after a test.
Use this when your tests contain transactions.
This is slower than ``TestCase`` but faster than ``IsolatedTestCase``.
Note that usage of this does not guarantee that auto-number-pks will be reset to 1.
"""
# async def asyncSetUp(self) -> None:
# _restore_default()
async def asyncTearDown(self) -> None:
Tortoise._connections = self._connections.copy()
# TODO: This is a naive implementation: Will fail to clear M2M and non-cascade foreign keys
for app in Tortoise.apps.values():
for model in app.values():
await model.all().delete()
class TransactionTestContext:
__slots__ = ("connection", "connection_name", "token")
def __init__(self, connection) -> None:
self.connection = connection
self.connection_name = connection.connection_name
async def __aenter__(self):
current_transaction = current_transaction_map[self.connection_name]
self.token = current_transaction.set(self.connection)
if hasattr(self.connection, "_parent"):
self.connection._connection = await self.connection._parent._pool.acquire()
await self.connection.start()
return self.connection
async def __aexit__(self, exc_type, exc_val, exc_tb) -> None:
await self.connection.rollback()
if hasattr(self.connection, "_parent"):
await self.connection._parent._pool.release(self.connection._connection)
current_transaction_map[self.connection_name].reset(self.token)
class TestCase(IsolatedTestCase):
"""
An asyncio capable test class that will ensure that each test will be run at
separate transaction that will rollback on finish.
This is a fast test runner. Don't use it if your test uses transactions.
"""
def run(self, result=None):
self._setupAsyncioLoop()
try:
_restore_default()
self.__db__ = Tortoise.get_connection("models")
if self.__db__.capabilities.supports_transactions:
connection = self.__db__._in_transaction().connection
connection_name = connection.connection_name
current_transaction = current_transaction_map[connection_name]
token = current_transaction.set(connection)
if hasattr(connection, "_parent"):
pool = connection._parent._pool
connection._connection = self._asyncioTestLoop.run_until_complete(pool.acquire())
self._asyncioTestLoop.run_until_complete(connection.start())
result = super().run(result)
self._asyncioTestLoop.run_until_complete(connection.rollback())
if hasattr(connection, "_parent"):
self._asyncioTestLoop.run_until_complete(connection._parent._pool.release(connection._connection))
current_transaction_map[connection_name].reset(token)
else:
result = super().run(result)
return result
finally:
self._tearDownAsyncioLoop()
async def asyncSetUp(self) -> None:
pass
async def asyncTearDown(self) -> None:
if self.__db__.capabilities.supports_transactions:
_restore_default()
else:
await super().asyncTearDown()
class TestCase2(IsolatedTestCase):
"""
An asyncio capable test class that will ensure that each test will be run at
separate transaction that will rollback on finish.
This is a fast test runner. Don't use it if your test uses transactions.
"""
async def _asyncCallWithTransactionContext(self, func, /, *args, **kwargs):
self.__db__ = Tortoise.get_connection("models")
if self.__db__.capabilities.supports_transactions:
connection = self.__db__._in_transaction().connection
async with TransactionTestContext(connection):
return await func(*args, **kwargs)
else:
return await func(*args, **kwargs)
async def _callWithTransactionContext(self, func, /, *args, **kwargs):
return await self._asyncCallWithTransactionContext(func, *args, **kwargs)
def _callAsync(self, func, /, *args, **kwargs):
assert self._asyncioTestLoop is not None
assert inspect.iscoroutinefunction(func)
ret = self._callWithTransactionContext(func, *args, **kwargs)
fut = self._asyncioTestLoop.create_future()
self._asyncioCallsQueue.put_nowait((fut, ret))
return self._asyncioTestLoop.run_until_complete(fut)
def _callMaybeAsync(self, func, /, *args, **kwargs):
assert self._asyncioTestLoop is not None
if inspect.iscoroutinefunction(func):
return self._callAsync(func, *args, **kwargs)
else:
return func(*args, **kwargs) |
Should I make a draft PR? |
Sqlite/Mysql/odbc drivers all wrap an existing library in a thread, so this is something we need to support properly. (somehow) I have not been able to solve that issue either. Looking at your proposed code, I have to say it looks a lot less hacky than what we currently have, but as long as the Your testcase2 seems to be wrapping the test in what is basically the same as I think you can create a proposed PR, and we then try and get this fixed? Unfortunately we still have to support Python3.7 for a while, and there we can just use the old testrunner for now? (Unless there is a backport?) |
I think the tests of the whole project should be ruined with plain unittest to serve as an example for people approaching the library, for this reason I would like to remove the whole hack of global variables in the initializer. I think we can get them to behave, my main issue is that the Would prefer to not override the logic of
Yes that something that I will be exploring this weekend.
Yes, just a simple copy and paste with a conditional import would do the trick. |
What do you mean by should be ruined?
Would be nice if we can get it to behave whist keeping test performance good. Because I really like being able to run the basic test suite in 4 seconds, instead of 4 minutes. |
Sorry, a non native english speaker mistake, I meant ran. I typed runned which my computer corrected in ruined.
Yes I see your point |
Makes so much more sense now 🤣 Don't apologise for non-native english speaker. I think your's is pretty good. My wife keeps on correcting my english 🤷♂️ |
Great work you guys are doing (could only understand 50% of what was being said). Sorry to butt in but are there any plans or progress regarding testing for python 3.8? Just wanted to know if this conversation is still ongoing or if you plan on releasing a dev version (or maybe a hack) for 3.8 users. Cheers to you guys! |
Hello @dropkickdev thanks for showing interest in this, My idea was to remove the global variable for setup, I did not get back to it, my bad. There is already some work here but it is still incomplete, I hope to get back to it soon. Last but not least, I am going to start some projects with tortoise so I will probably get back to it |
Thanks for replying so soon @WisdomPill. Yes, this very much caught my attention and yes those global variables surprised me so much it made me choke on my coffee (true story). I haven't made a fork yet but merely a clone so I could analyze it offline. I really appreciate the effort you've put into this especially since tests are important for upgrades. @WisdomPill would you recommend I use your fork instead of the main line for python 3.8? Tortoise works fine it's just the testing that's got me rattled. |
I had some issues with transactional tests because of context vars and I did not complete, I think there is still work to be done. |
Do you remember what error message it was while working on transactions? Maybe I can take a look. |
You know what, I'll go ahead and fork your fork. 🤓 Although I sincerely hope they do something about this. Things can get really scary without any testing. Hey @grigi @WisdomPill if any of you are in the Philippines hit me up. We can get a beer. |
No, I do not remember, I think a lot has to be rewritten, we can work on the same PR instead of forking? P.S. Thanks for the invitation but I am far away from Philippines, very far, on the other side of the planet in europe |
Hi, Shouldn't asynctest be added to tortoise-orm's main dependencies? I see that it is included in the dev dependencies, but if someone wants to write tests for their app that uses tortoise, it needs to be installed. Thanks for the great work! |
Hi guys. class IsolatedTestCase(IsolatedAsyncioTestCase):
"""
An asyncio capable test class that will ensure that an isolated test db
is available for each test.
Use this if your test needs perfect isolation.
Note to use ``{}`` as a string-replacement parameter, for your DB_URL.
That will create a randomised database name.
It will create and destroy a new DB instance for every test.
This is obviously slow, but guarantees a fresh DB.
If you define a ``tortoise_test_modules`` list, it overrides the DB setup module for the tests.
"""
tortoise_test_modules: List[str] = []
db_url: str = ''
async def asyncSetUp(self) -> None:
config = generate_config(self.db_url, app_modules={'models': self.tortoise_test_modules},
testing=True, connection_label='models')
await Tortoise.init(config, _create_db=True)
await Tortoise.generate_schemas(safe=False)
self._connections = Tortoise._connections.copy()
async def asyncTearDown(self) -> None:
Tortoise._connections = self._connections.copy()
await Tortoise._drop_databases() Everything is good while I use strawberry(Graphql)
Do you have any idea? |
I used xhttp instead of |
But for this we should give up |
see #977 |
First and foremost, congrats, I really like this library and looking forward to contribute and follow the path of its evolution. I embraced this library mostly because it is
async
and it resemblesdjango
so it was very easy to get started with.Is your feature request related to a problem? Please describe.
I am using tortoise with python 3.8,
as of python 3.8 asyncio unittesting is possible without asynctest.
https://docs.python.org/3/library/unittest.html#unittest.IsolatedAsyncioTestCase
There are also
AsyncMock
andpatch
supports async methods.Describe the solution you'd like
I would like not to include
asynctest
when using python3.8 and above.Describe alternatives you've considered
I have considered no other alternatives, but I am open to consider other alternatives.
Additional context
I have been using a work around by copying some code from
IsolatedTestCase
I am open to open a PR with the fixes.
Another question, why the testing modules are in
contrib
?Testing is vital to every piece of software, is testing not fully supported or you are planning to change the api?
The text was updated successfully, but these errors were encountered: