[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow reusing existing tensors for outputs of TF_SessionRun() #46711

Open
shoelzer opened this issue Jan 26, 2021 · 2 comments
Open

Allow reusing existing tensors for outputs of TF_SessionRun() #46711

shoelzer opened this issue Jan 26, 2021 · 2 comments
Assignees
@rohan100jain
Labels
comp:apis Highlevel API related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower type:feature Feature requests

Comments

@shoelzer
Copy link

In the C API, TF_SessionRun() allocates new output tensors on every call. This leads to frequent allocation/deallocation and memory copies in some situations. It would be useful to be able to reuse existing tensors for output.

The current API takes TF_Tensor** output_values and will update that variable to point to newly allocated output tensors. I propose that if output_values points to existing tensors, those tensors should be used for output instead of allocating new tensors.
@shoelzer shoelzer added the type:feature Feature requests label Jan 26, 2021
@amahendrakar amahendrakar added the comp:apis Highlevel API related issues label Jan 28, 2021
@amahendrakar amahendrakar assigned ymodak and unassigned amahendrakar Jan 28, 2021
@ymodak ymodak assigned saxenasaurabh and unassigned ymodak Jan 28, 2021
@ymodak ymodak added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jan 28, 2021
@DamRsn
Copy link
DamRsn commented Nov 1, 2021

Hello,

Any news on this?

I also believe that the overhead of memory allocation should be avoided when calling TF_SesssionRun() multiple times. I'm using the Tensorflow C API (2.6) for inference in the context of real-time audio processing, and malloc is not a real-time safe operation. Having a way to avoid this memory allocation is crucial for I think numerous use cases.

This issue has been pointed more than 2 years ago for Tensorflow 1.13 (#29733).

The solution suggested by @shoelzer seems good to me. There might also be other ways to implement it.

@hoyden
Copy link
hoyden commented Nov 15, 2021

Same problem in C++ API, did anyone solve it ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rohan100jain rohan100jain

Labels
comp:apis Highlevel API related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower type:feature Feature requests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@rohan100jain @saxenasaurabh @shoelzer @hoyden @ymodak @DamRsn @amahendrakar