@VoVAllen ,
When tried executing the given code AttributeError: module 'tensorflow' has no attribute 'numpy' is faced, kindly find the gist of colab. Thanks!

@oanush Sorry I mistakenly write the code, it should be a.numpy() instead of tf.numpy()

The following code should work

import tensorflow as tf
import numpy as np
print(tf.__version__) # 2.0.0
tf.executing_eagerly() # True

a = tf.constant([3, 4]).cpu()
b = a.numpy()
b[0] = 1
print(a)
# [3, 4], Expected to be [1, 4] if share memory with numpy array
print(b)
# [1, 4]

As described in https://www.tensorflow.org/tutorials/customization/basics#numpy_compatibility, they should share the underlying memorys.

Issue replicating with TF-2.0, kindly find the gist of collab. Thanks!

@VoVAllen,
Below Text from the Tutorial doesn't say that Numpy Array and Tensors share the Underlying Memory always, but it was a conditional statement ("if possible" was mentioned).

These conversions are typically cheap since the array and tf.Tensor share the underlying memory representation, if possible. However, sharing the underlying representation isn't always possible since the tf.Tensor may be hosted in GPU memory while NumPy arrays are always backed by host memory, and the conversion involves a copy from GPU to host memory.

It is evident from the Implementation of Tensor.numpy()] as well that Sharing of Underlying Memory is not Mandatory.

Below sentence states that,

TODO(ashankar,agarwal): Perhaps this should NOT reference the underlying
buffer but instead always explicitly copy? Note that currently it may or may
not copy based on whether the numpy data is properly aligned or not.

Considering all these observations, as per my understanding, this behavior is expected. Please let me know your thoughts. Thanks!

Hi,

Thanks for your reply.

Actually I think I found the bug here. At here

If it is a numpy.ndarray, why not directly return maybe_arr? The logic here says return a copy of numpy.ndarray, or directly return something which is not a numpy.ndarray. It's weird.

And I checked maybe_arr shares memory with original tensor, which as expected.

By the following code I got what I expected:

import tensorflow as tf
import numpy as np
print(tf.__version__) # 2.0.0
tf.executing_eagerly() # True

a = tf.constant([3, 4]).cpu()
b = a._numpy() # Change from numpy() to _numpy()
b[0] = 1
print(a)
# [1, 4]
print(b)
# [1, 4]

At least there're some inconsistency between the doc and the behavior,

I checked some history commit (7caec68), the copy behavior seems intentional. The comment at numpy function seems outdated. I couldn't find way that share memory between tensorflow and numpy, no matter from numpy to tf or from tf to numpy.

At https://github.com/tensorflow/tensorflow/blame/5278b8509e2cd1b2847315db46fc0f958824cfce/tensorflow/python/framework/ops.py#L709, numpy is zero-copyed from tf.

The next commit (ca1b54a?diff=split), fixed that the .cpu() operation should not be recorded. To fix this, this author copied tensor to create a new one which is not recorded. This fix seems not ideal, since avoiding recording the operation doesn't have to copy a new one. From here, it becomes the copy behavior.

Later at

@superbobry Sorry to bother you. Could you comment a bit on this issue, that at what condition tf tensor can share memory with numpy? Also could you say a bit more about the reference counting problem mentioned above? Many thanks!

NumPy arrays are mutable whereas tensors aren't, therefore .numpy() has to copy to avoid breaking the runtime's invariants. In theory, a NumPy array could be flagged as non-writable (via a.flags["WRITABLE"]), but unfortunately a lot of internal (and I assume external) users already rely on the mutability of .numpy().

Two caveats:

Tensor supports the buffer interface, so you could get a readonly NumPy array via

>>> t = tf.constant([42])
>>> a = np.asarray(memoryview(t))
>>> a
array([42], dtype=int32)

Passing a CPU tensor directly to any NumPy APIs is zero-copy, because NumPy uses buffer interface behind the scenes (but NumPy will allocate a new array for the result):

>>> np.square(t)
array([1764], dtype=int32)

Thanks a lot for the explanation. This makes much more sense to me.

Are you satisfied with the resolution of your issue?
Yes
No