[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

psutil error #35

Closed
hkoohy opened this issue Aug 23, 2017 · 10 comments
Closed

psutil error #35

hkoohy opened this issue Aug 23, 2017 · 10 comments

Comments

@hkoohy
Copy link
hkoohy commented Aug 23, 2017

I just have scanpy 0.2.7 and am trying to produce bpmc3 results. BUT right at the beginning (sc.read()) the following error! I will appreciate your help.
thanks

`--------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
in ()
2 filename_genes = '/ifs/projects/proj077/backup/public_data/scanpy_tutorials_data/PBMC3K/filtered_gene_bc_matrices/hg19/genes.tsv'
3 filename_barcodes = '/ifs/projects/proj077/backup/public_data/scanpy_tutorials_data/PBMC3K/filtered_gene_bc_matrices/hg19/barcodes.tsv'
----> 4 adata = sc.read(filename_data, cache=True).transpose()
5 adata.var_names = np.genfromtxt(filename_genes, dtype=str)[:, 1]
6 adata.smp_names = np.genfromtxt(filename_barcodes, dtype=str)

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/readwrite.py in read(filename_or_filekey, sheet, ext, delimiter, first_column_names, backup_url, return_dict, cache)
73 if is_filename(filename_or_filekey):
74 data = read_file(filename_or_filekey, sheet, ext, delimiter,
---> 75 first_column_names, backup_url, cache)
76 if isinstance(data, dict):
77 return data if return_dict else AnnData(data)

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/readwrite.py in read_file(filename, sheet, ext, delimiter, first_column_names, backup_url, cache)
364 os.makedirs(os.path.dirname(filename_cache))
365 # write for faster reading when calling the next time
--> 366 write_dict_to_file(filename_cache, ddata, sett.file_format_data)
367 return ddata
368

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/readwrite.py in write_dict_to_file(filename, d, ext)
771 d_write[key] = value
772 # now open the file
--> 773 wait_until_file_unused(filename) # thread-safe writing
774 if ext == 'h5':
775 with h5py.File(filename, 'w') as f:

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/readwrite.py in wait_until_file_unused(filename)
935
936 def wait_until_file_unused(filename):
--> 937 while (filename in get_used_files()):
938 time.sleep(1)
939

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/readwrite.py in get_used_files()
919 def get_used_files():
920 """Get files used by processes with name scanpy."""
--> 921 loop_over_scanpy_processes = (proc for proc in psutil.process_iter()
922 if proc.name() == 'scanpy')
923 filenames = []

AttributeError: module 'psutil' has no attribute 'process_iter'

`

@falexwolf
Copy link
Member
falexwolf commented Aug 23, 2017

That is evidently a problem of psutil; do you have an old version of it? I have tested with 5.2.2 and 5.1.2. Earlier, psutil seemed to have had a different convention. But both is not related to Scanpy.

@falexwolf
Copy link
Member
falexwolf commented Aug 23, 2017

Does running pip install psutil --upgrade help? If yes, I will update the requirements with a minimal version of psutil so that others don't run into the same problem.

@falexwolf
Copy link
Member

Hm, I researched a bit more. psutil doesn't seem to cause problems and also, this has not been a problem within Scanpy for any user up to now. If you start a terminal with python and type

import psutil
psutil.process_iter()

does this throw an error? I'd really like to know what's going on.

If you want a quick fix; you can simply comment out line 773 in your file /ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/readwrite.py; this should cause no problem for your applications.

@falexwolf
Copy link
Member

Btw: Scanpy now has an initial documentation at https://scanpy.readthedocs.io.

@hkoohy
Copy link
Author
hkoohy commented Aug 24, 2017

Hi Alex,
The psutil issue by updating it has apparently gone away, however later on when I call sc.tl.louvain(adata_corrected, n_neighbors=10, resolution=1.3, recompute_graph=True)
, I get the following error. The igraph I am using is V 0.1.11.
Many thanks
Hashem
`DeprecationWarning Traceback (most recent call last)
in ()
1
----> 2 sc.tl.louvain(adata_corrected, n_neighbors=10, resolution=1.3, recompute_graph=True)
3

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/tools/louvain.py in louvain(adata, n_neighbors, resolution, n_pcs, random_state, flavor, directed, recompute_pca, recompute_distances, recompute_graph, n_dcs, n_jobs, copy)
78 directed = False
79 if not directed: logg.m(' using the undirected graph', v=4)
---> 80 g = utils.get_igraph_from_adjacency(adjacency, directed=directed)
81 if flavor == 'vtraag':
82 import louvain

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/scanpy/utils.py in get_igraph_from_adjacency(adjacency, directed)
41 def get_igraph_from_adjacency(adjacency, directed=None):
42 """Get igraph graph from adjacency matrix."""
---> 43 import igraph as ig
44 sources, targets = adjacency.nonzero()
45 weights = adjacency[sources, targets]

/ifs/devel/hashem/sw-v1/conda/lib/python3.6/site-packages/igraph/init.py in ()
6 license = "MIT"
7
----> 8 raise DeprecationWarning("To avoid name collision with the igraph project, "
9 "this visualization library has been renamed to "
10 "'jgraph'. Please upgrade when convenient.")

DeprecationWarning: To avoid name collision with the igraph project, this visualization library has been renamed to 'jgraph'. Please upgrade when convenient.`

@falexwolf
Copy link
Member
falexwolf commented Aug 24, 2017

This is again a configuration problem. You don't have http://igraph.org/python/ installed. That's a library with thousands of users and citations. It has a very powerful and fast C++ core that allows treating dataset sizes with a million cells.

I realize that I misspecified this in Scanpy's automatic installation in the requirements file. I just updated this and will push it to the master branch.

You simply need to type pip install python-igraph and then everything should work.

@hkoohy
Copy link
Author
hkoohy commented Aug 24, 2017

I think I have the most recent version of python-graph installed ( python-igraph 0.7.1.post6 ).
pip install python-igraph doesnt solve the issue since it is already updated.

@hkoohy
Copy link
Author
hkoohy commented Aug 24, 2017

I think this was happening because of confusion between igraph and python-igraph, by un-installing both and reinstalling just python-igraph, the issue was solved. Interesting enough installation with conda didnt work, only installation with pip worked.
thanks again.

@falexwolf
Copy link
Member

Sorry for being offline for a few hours and great that you could resolve it. In the next version, this confusion will not appear again.

Even though, obviously, any Scanpy release reproduces all examples on https://github.com/theislab/scanpy_usage, we're still at an early stage in the package. Things are progressing very fast and structure and maintenance of the package are becoming more and more professional. Also, soon, many new features and examples where other Python packages are used will be added.

Thank you for a bit of patience at this stage.
Alex

PS: We now have an initial version of the documentation: https://scanpy.readthedocs.io.

@hkoohy
Copy link
Author
hkoohy commented Aug 24, 2017

No worries and thank you for usually very prompt suggestions.
The idea that scanpy can handle many cells efficiently is great and therefore I have been trying it in a computing cluster (and not my local machine) for the future usage. This in turn makes configuration just a bit more difficult.
Looking forward to a more stable version with more added function.
Thank you
Hashem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants