[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Edit the requests configuration pool_maxsize variable when initializing client #521

Open
lautarovc opened this issue Apr 7, 2021 · 1 comment

Comments

@lautarovc
Copy link
lautarovc commented Apr 7, 2021
  • Algolia Client Version: 2.4.0
  • Language Version: Python 3.9.1

Description

I'm replicating a dataset (60k records) with 100 threads (Python's builtin ThreadExecutor) at a time. And I find that from time to time I keep getting these warnings from the 'requests' library: "Connection pool is full, discarding connection: xxxxxxxx.algolia.net".

The ability to edit the pool_maxsize configuration variable when initializing the search client would solve this kind of issues, perhaps even allowing to edit other requests configurations.

Steps To Reproduce

Initialize a ThreadExecutor of 100 max workers.
Create a function that uses 'save_object' to store a record in a predefined index.
Loop through thousands of records, for each record:
-Add the function that saves the record with the record data to the executor

You should get the warning every ~1000 records.

@rgs258
Copy link
rgs258 commented Jan 6, 2023

I'm seeing the same thing. As a workaround, I'm initializing the client in the callable. Would love to see this taken up!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants