[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Odd error awaiting a Future #81353

Closed
dimaqq mannequin opened this issue Jun 6, 2019 · 11 comments
Closed

Odd error awaiting a Future #81353

dimaqq mannequin opened this issue Jun 6, 2019 · 11 comments

Comments

@dimaqq
Copy link
Mannequin
dimaqq mannequin commented Jun 6, 2019
BPO 37172
Nosy @asvetlov, @dimaqq, @1st1

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-06-06.10:29:59.344>
labels = ['3.7', '3.8', '3.9', 'expert-asyncio']
title = 'Odd error awaiting a Future'
updated_at = <Date 2019-06-10.10:18:40.184>
user = 'https://github.com/dimaqq'

bugs.python.org fields:

activity = <Date 2019-06-10.10:18:40.184>
actor = 'Dima.Tisnek'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2019-06-06.10:29:59.344>
creator = 'Dima.Tisnek'
dependencies = []
files = []
hgrepos = []
issue_num = 37172
keywords = []
message_count = 9.0
messages = ['344798', '344800', '345017', '345100', '345102', '345111', '345114', '345115', '345118']
nosy_count = 3.0
nosy_names = ['asvetlov', 'Dima.Tisnek', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue37172'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

@dimaqq
Copy link
Mannequin Author
dimaqq mannequin commented Jun 6, 2019

Let's start with correct code:

import asyncio


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    global g1
    g1 = asyncio.Future()
    await asyncio.gather(reader(), writer())

asyncio.run(test())

No error, as expected.

Now let's mess it up a bit:

import asyncio

g1 = asyncio.Future()


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    await asyncio.gather(reader(), writer())

asyncio.run(test())

Fails with RuntimeError ... attached to a different loop

The error makes sense, although it's sad that I can't create global futures / there was no even loop when Future was creates, it was not a *different* event loop / maybe I wish .run() didn't force a new event loop?

A nit (IMO), but I can live with it.

Let's mess the code up a bit more:

import asyncio

g1 = asyncio.Future()


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    await asyncio.gather(reader(), reader(), writer())

asyncio.run(test())

RuntimeError: await wasn't used with future

What?
That's really confusing!
The only difference is that there are now 2 readers running in parallel.

The actual exception comes from asyncio.Future.__await__ after a yield.
I'm not sure how to fix this...

@dimaqq dimaqq mannequin added 3.7 (EOL) end of life 3.8 only security fixes topic-asyncio labels Jun 6, 2019
@asvetlov
Copy link
Contributor
asvetlov commented Jun 6, 2019

Global future objects (and global asyncio objects in general) don't work with asyncio.run(). The lifecycle of these objects should be closer than loop but asyncio.run() creates a loop during execution.

I think this is a good design, we have a plan to deprecate and eventually drop all old-times mess that allows confusions.

All three of your problems are because you use a global future which is implicitly attached to the different loop.

Also I'd like to note that futures are low-level API, which is very delicate and error-prone. Futures are crucial for libraries building (e.g. aiohttp uses them a lot) but working with futures in application code is an explicit sign of bad design.

@terryjreedy terryjreedy added the 3.9 only security fixes label Jun 7, 2019
@terryjreedy terryjreedy changed the title Odd error awating a Future Odd error awaiting a Future Jun 7, 2019
@terryjreedy
Copy link
Member

Dima, unless you want to make a specific doc change suggestion, I think this should be closed. The planned code changes will be on other issues.

@dimaqq
Copy link
Mannequin Author
dimaqq mannequin commented Jun 10, 2019

Hi Terry,

Yes, I have a specific suggestion:

The error RuntimeError: await wasn't used with future is misleading.
I'm not sure if changing error text is enough.
I think that Future.await should be fixed; I think that await f should be idempotent, that the same exception should be raised in single and double await examples.

@terryjreedy
Copy link
Member

OK. I am quitting here because asyncio and futures are outside my expertise.

@asvetlov
Copy link
Contributor

Not sure what "idempotency fix" means in the context of Future objects.

Could you describe desired behavior or, even better, provide a pull request that demonstrates your desire?

@dimaqq
Copy link
Mannequin Author
dimaqq mannequin commented Jun 10, 2019

I think that if a Future is abused, the following two should return *same* error:

async def test():
    await asyncio.gather(reader(), writer())

-vs-

async def test():
    await asyncio.gather(reader(), reader(), writer())

@asvetlov
Copy link
Contributor

Agree, it would be fine.
So the behavior is not changed but the error text should be clear, right?

@dimaqq
Copy link
Mannequin Author
dimaqq mannequin commented Jun 10, 2019

Yes, I think that would be sufficient!

I don't know about implementation though, as there's some magic in the way Future.__await__ tries to guess how it was called. I hope one day that magic could go away, but I neither understand the origin nor the mechanics of it, so... My gut tells me that fixing the error involves digging into that magic.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@kumaraditya303 kumaraditya303 removed 3.9 only security fixes 3.8 only security fixes 3.7 (EOL) end of life labels Oct 18, 2022
@kumaraditya303
Copy link
Contributor

This is definitely one of the obscure edge cases with mixing global futures and different loop. I checked and this bug is only present in the C implementation so a bug in _asynciomodule.c.

@kumaraditya303
Copy link
Contributor

Let's not spend more time on this, the easier way to avoid this is to just avoid using global futures which isn't recommended either.

@kumaraditya303 kumaraditya303 closed this as not planned Won't fix, can't repro, duplicate, stale Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

3 participants