[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot seek on write only tf.gfile.GFile #32122

Open
eirism opened this issue Aug 30, 2019 · 12 comments
Open

Cannot seek on write only tf.gfile.GFile #32122

eirism opened this issue Aug 30, 2019 · 12 comments
Labels
comp:apis Highlevel API related issues TF 2.11 Issues related to TF 2.11 type:bug Bug

Comments

@eirism
Copy link
eirism commented Aug 30, 2019

System information

  • Have I written custom code: Yes
  • OS Platform and Distribution: Linux Ubuntu 18.04
  • TensorFlow installed from: binary
  • TensorFlow version: v1.14.0-rc1-22-gaf24dc9 1.14.0
  • Python version: 3.6

Describe the current behavior
Calling seek() on a tf.gfile.GFile opened in write only mode raises tensorflow.python.framework.errors_impl.PermissionDeniedError.

Describe the expected behavior
GFile should support the Python IO semantics that supports seeking on a write only file.

More generally it would be preferable if GFile followed the API of Python's io.IOBase.

Code to reproduce the issue

import tensorflow as tf

with tf.io.gfile.GFile('test.txt', 'w') as f:
    f.seek(0)

Other info / logs

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/VENV/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/VENV/lib/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 146, in seek
    self._preread_check()
  File "/VENV/lib/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 82, in _preread_check
    "File isn't open for reading")
tensorflow.python.framework.errors_impl.PermissionDeniedError: File isn't open for reading
@mihaimaruseac
Copy link
Collaborator

Related to #32090, right?

@mihaimaruseac mihaimaruseac self-assigned this Aug 30, 2019
@eirism
Copy link
Author
eirism commented Aug 30, 2019

It is at least related in the sense that GFile does not behave as normal Python file like objects.

This specific bug seems to be caused by an assumption that only readable files can be seeked.
It seems that tell() had the same behaviour until this commit.

@mihaimaruseac
Copy link
Collaborator

Well, yes, the API is incomplete.

My work on modular filesystems will first separate all filesystems into DSOs and then write a test for functionality requirements and then we can start completing the API and checking in every system to make sure it works. Maybe even fuzz the entire thing

@372046933
Copy link
Contributor

test_seek_v1 works but test_seek_v2 not

def test_seek_v1():
  with open('file.txt', 'w+') as f:   # create a new file or truncates it
    f.write("test 1\n")
    f.write("test 2\n")
    f.write("test 3\n")             # now the file pointer is at the end
    f.seek(0)                       # move the file pointer to the beginning
    lines = f.read()                # read it, now we can read!
    print(lines)                    # print it

def test_seek_v2():
  with tf.io.gfile.GFile('file.txt', 'w+') as f:   # create a new file or truncates it
    f.write("test 1\n")
    f.write("test 2\n")
    f.write("test 3\n")             # now the file pointer is at the end
    f.seek(0)                       # move the file pointer to the beginning
    lines = f.read()                # read it, now we can read!
    print(lines)                    # print it

@mihaimaruseac mihaimaruseac removed their assignment Nov 2, 2022
@mihaimaruseac
Copy link
Collaborator

I think test_seek_v2 is actually correct. You are opening the file only for writing so of course you should not be able to read from it

@eirism
Copy link
Author
eirism commented Nov 3, 2022

The + argument to the built in open() function means that the file should be opened for updating (reading and writing), see the docs. So the w+ mode of the built in open() function means to open the file for reading and writing and truncating it first. I do not see any documentation of the mode parameter for tf.io.gfile.GFile in the TF docs, so I can only assume that the semantics are supposed to be the same as the mode parameter for the built in open().

This issue is about calling .seek() on a write only file (w), not about the behavior of the w+ mode. So test_seek_v1 and test_seek_v2 are not relevant for this issue, but that behavior might be a separate issue.

@372046933
Copy link
Contributor

test_seek_v2 open mode is reading and writing

mode: one of `r`, `w`, `a`, `r+`, `w+`, `a+`. Append `b` for bytes mode.

@mihaimaruseac
Copy link
Collaborator

My bad, I misread the + meaning.

@mohantym mohantym self-assigned this Jan 14, 2023
@mohantym mohantym added TF 1.14 for issues seen with TF 1.14 comp:apis Highlevel API related issues type:bug Bug labels Jan 14, 2023
@mohantym
Copy link
Contributor

@eirism !
I can replicate this in 2.11 version. Attached gist in 2.11 for reference.
Thank you!

@mohantym mohantym added TF 2.11 Issues related to TF 2.11 and removed TF 1.14 for issues seen with TF 1.14 labels Jan 16, 2023
@mohantym mohantym removed their assignment Jan 16, 2023
@sushreebarsa
Copy link
Contributor

@eirism The provided code will trigger an error because you're attempting to call f.seek(0) on a file opened in write-only mode ('w'). Seeking is only supported for files opened in read or read/write modes.
Could you try to use it as follows;

with tf.io.gfile.GFile('test.txt', 'r+') as f:
  f.seek(0)
  # Your code that includes writing and seeking

Thank you!

@sushreebarsa sushreebarsa self-assigned this Dec 15, 2023
@sushreebarsa sushreebarsa added the stat:awaiting response Status - Awaiting response from author label Dec 15, 2023
@mihaimaruseac
Copy link
Collaborator

@sushreebarsa I had the same comment a while ago and got #32122 (comment) as reply (which makes sense)

@google-ml-butler google-ml-butler bot removed the stat:awaiting response Status - Awaiting response from author label Dec 15, 2023
@sushreebarsa sushreebarsa removed their assignment Dec 16, 2023
@eirism
Copy link
Author
eirism commented Dec 22, 2023

@sushreebarsa Normal Python file IO supports .seek() on files opened in write-only mode ('w'), so I expect GFile to also support it. In normal Python file IO .seek() moves the file object position and allows you to overwrite already written data. The following code using normal Python file IO will write "Hello" to the text file "test.txt" by using .seek() to overwrite the first character written to file:

with open("test.txt", "w") as f:
    f.write("hello")
    f.seek(0)  # Seek to start of file to overwrite characters from the start
    f.write("H")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:apis Highlevel API related issues TF 2.11 Issues related to TF 2.11 type:bug Bug
Projects
None yet
Development

No branches or pull requests

5 participants