[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NonMatchingChecksumError for celeb_a #3150

Closed
ghost opened this issue Apr 8, 2021 · 3 comments
Closed

NonMatchingChecksumError for celeb_a #3150

ghost opened this issue Apr 8, 2021 · 3 comments
Labels
bug Something isn't working

Comments

@ghost
Copy link
ghost commented Apr 8, 2021

Short description
NonMatchingChecksumError while trying to load celeb_a

Environment information

  • Operating System: Ubuntu 20.10

  • Python version: 3.8.2

  • tensorflow-datasets/tfds-nightly version: 4.2.0

  • tensorflow/tf-nightly version: 2.4.1

  • Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?
    Yes

Reproduction instructions
Followed the "load a dataset" instructions at TensoFlow Datasets

import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf

import tensorflow_datasets as tfds

ds = tfds.load('celeb_a', split='train', shuffle_files=True)
assert isinstance(ds, tf.data.Dataset)
print(ds)

Link to logs

In [1]: import matplotlib.pyplot as plt
   ...: import numpy as np
   ...: import tensorflow as tf
   ...: 
   ...: import tensorflow_datasets as tfds
   ...: 
   ...: ds = tfds.load('celeb_a', split='train', shuffle_files=True)
   ...: assert isinstance(ds, tf.data.Dataset)
   ...: print(ds)
2021-04-08 21:12:31.178377: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
2021-04-08 21:12:32.546738: W tensorflow/core/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "Not found: Could not locate the credentials file.". Retrieving token from GCE failed with "Failed precondition: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Couldn't resolve host 'metadata'".
Downloading and preparing dataset Unknown size (download: Unknown size, generated: Unknown size, total: Unknown size) to /home/ashvith/tensorflow_datasets/celeb_a/2.0.1...
Dl Size...: 0 MiB [00:02, ? MiB/s]████████████████████████████████████████████████████████████████████████████████████████████████                                    | 3/4 [00:02<00:00,  1.44 url/s]
Dl Completed...:  75%|████████████████████████████████████████████████████████████████████████████████████████████████████████████                                    | 3/4 [00:02<00:00,  1.26 url/s]
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
<ipython-input-1-a9209b5c03a5> in <module>
      5 import tensorflow_datasets as tfds
      6 
----> 7 ds = tfds.load('celeb_a', split='train', shuffle_files=True)
      8 assert isinstance(ds, tf.data.Dataset)
      9 print(ds)

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/load.py in load(name, split, data_dir, batch_size, shuffle_files, download, as_supervised, decoders, read_config, with_info, builder_kwargs, download_and_prepare_kwargs, as_dataset_kwargs, try_gcs)
    326   if download:
    327     download_and_prepare_kwargs = download_and_prepare_kwargs or {}
--> 328     dbuilder.download_and_prepare(**download_and_prepare_kwargs)
    329 
    330   if as_dataset_kwargs is None:

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/dataset_builder.py in download_and_prepare(self, download_dir, download_config)
    442           # Old version of TF are not os.PathLike compatible
    443           with tf_compat.mock_gfile_pathlike():
--> 444             self._download_and_prepare(
    445                 dl_manager=dl_manager,
    446                 download_config=download_config,

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/dataset_builder.py in _download_and_prepare(self, dl_manager, download_config)
   1145       else:
   1146         optional_pipeline_kwargs = {}
-> 1147       split_generators = self._split_generators(  # pylint: disable=unexpected-keyword-arg
   1148           dl_manager, **optional_pipeline_kwargs
   1149       )

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/image/celeba.py in _split_generators(self, dl_manager)
    126 
    127   def _split_generators(self, dl_manager):
--> 128     downloaded_dirs = dl_manager.download({
    129         "img_align_celeba": IMG_ALIGNED_DATA,
    130         "list_eval_partition": EVAL_LIST,

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py in download(self, url_or_urls)
    547     # Add progress bar to follow the download state
    548     with self._downloader.tqdm():
--> 549       return _map_promise(self._download, url_or_urls)
    550 
    551   def iter_archive(

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py in _map_promise(map_fn, all_inputs)
    775   """Map the function into each element and resolve the promise."""
    776   all_promises = tf.nest.map_structure(map_fn, all_inputs)  # Apply the function
--> 777   res = tf.nest.map_structure(lambda p: p.get(), all_promises)  # Wait promises
    778   return res

~/anaconda3/lib/python3.8/site-packages/tensorflow/python/util/nest.py in map_structure(func, *structure, **kwargs)
    657 
    658   return pack_sequence_as(
--> 659       structure[0], [func(*x) for x in entries],
    660       expand_composites=expand_composites)
    661 

~/anaconda3/lib/python3.8/site-packages/tensorflow/python/util/nest.py in <listcomp>(.0)
    657 
    658   return pack_sequence_as(
--> 659       structure[0], [func(*x) for x in entries],
    660       expand_composites=expand_composites)
    661 

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py in <lambda>(p)
    775   """Map the function into each element and resolve the promise."""
    776   all_promises = tf.nest.map_structure(map_fn, all_inputs)  # Apply the function
--> 777   res = tf.nest.map_structure(lambda p: p.get(), all_promises)  # Wait promises
    778   return res

~/anaconda3/lib/python3.8/site-packages/promise/promise.py in get(self, timeout)
    510         target = self._target()
    511         self._wait(timeout or DEFAULT_TIMEOUT)
--> 512         return self._target_settled_value(_raise=True)
    513 
    514     def _target_settled_value(self, _raise=False):

~/anaconda3/lib/python3.8/site-packages/promise/promise.py in _target_settled_value(self, _raise)
    514     def _target_settled_value(self, _raise=False):
    515         # type: (bool) -> Any
--> 516         return self._target()._settled_value(_raise)
    517 
    518     _value = _reason = _target_settled_value

~/anaconda3/lib/python3.8/site-packages/promise/promise.py in _settled_value(self, _raise)
    224             if _raise:
    225                 raise_val = self._fulfillment_handler0
--> 226                 reraise(type(raise_val), raise_val, self._traceback)
    227             return self._fulfillment_handler0
    228 

~/anaconda3/lib/python3.8/site-packages/six.py in reraise(tp, value, tb)
    701             if value.__traceback__ is not tb:
    702                 raise value.with_traceback(tb)
--> 703             raise value
    704         finally:
    705             value = None

~/anaconda3/lib/python3.8/site-packages/promise/promise.py in try_catch(handler, *args, **kwargs)
     85     # type: (Callable, Any, Any) -> Union[Tuple[Any, None], Tuple[None, Tuple[Exception, Optional[TracebackType]]]]
     86     try:
---> 87         return (handler(*args, **kwargs), None)
     88     except Exception as e:
     89         tb = exc_info()[2]

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py in <lambda>(dl_result)
    358 
    359     # Post-process the result
--> 360     return future.then(lambda dl_result: self._register_or_validate_checksums(  # pylint: disable=g-long-lambda
    361         url=url,
    362         path=dl_result.path,

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py in _register_or_validate_checksums(self, path, url, expected_url_info, computed_url_info, checksum_path, url_path)
    414       #   download). This is expected as it might mean the downloaded file
    415       #   was corrupted. Note: The tmp file isn't deleted to allow inspection.
--> 416       _validate_checksums(
    417           url=url,
    418           path=path,

~/anaconda3/lib/python3.8/site-packages/tensorflow_datasets/core/download/download_manager.py in _validate_checksums(url, path, computed_url_info, expected_url_info, force_checksums_validation)
    755         'https://www.tensorflow.org/datasets/overview#fixing_nonmatchingchecksumerror'
    756     )
--> 757     raise NonMatchingChecksumError(msg)
    758 
    759 

NonMatchingChecksumError: Artifact https://drive.google.com/uc?export=download&id=0B7EVK8r0v71pZjFTYXZWM3FlRnM, downloaded to /home/ashvith/tensorflow_datasets/downloads/ucexport_download_id_0B7EVK8r0v71pZjFTYXZWM3FlDDaXUAQO8EGH_a7VqGNLRtW52mva1LzDrb-V723OQN8.tmp.547e948087c644bf954aae797ea3a7f9/uc, has wrong checksum. Expected: UrlInfo(size=1.34 GiB, checksum='46fb89443c578308acf364d7d379fe1b9efb793042c0af734b6112e4fd3a8c74', filename='img_align_celeba.zip'). Got: UrlInfo(size=3.05 KiB, checksum='bcb287b2de2b4acb68642752c393b4b360640119a3dd45f92d25b324c4cba074', filename='uc').To debug, see: https://www.tensorflow.org/datasets/overview#fixing_nonmatchingchecksumerror

Expected behavior
celeb_a should download successfully.

Additional context
Using anaconda

@ghost ghost added the bug Something isn't working label Apr 8, 2021
@ghost
Copy link
Author
ghost commented Apr 9, 2021

The issue is no longer persistent. Looks like there was some problem with Google Drive probably.

@ghost ghost closed this as completed Apr 9, 2021
@ghost ghost reopened this Apr 20, 2021
@ghost
Copy link
Author
ghost commented Apr 20, 2021

Looks like the issue is up again.

@ghost
Copy link
Author
ghost commented Apr 20, 2021

#1482 maybe, so closing this issue

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

0 participants