-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
shmmap and shmvar does not exist in GDL #860
Comments
should be, er, straightforward for somebody with a modicum of ease in C... |
I am coming back to this old issue because I would need it a lot. I am writing programs for multi CPU machines and do not know how I can share the data between the processes without using shmmap and shmvar. Can anybody work on that or is there any workaround? |
Could you please provide an example of usage ? It is not clear for me, I never had to use that. I also suppose it is totally different from what we do on multi-cores machines ? When I use Slurm or PBS I reserve before the rum some resources, including both CPUs (cores) & RAM ... on a node .. Are such functions useful nowadays ? (it is a naive question) |
Let me show you an easy example. Open two terminals with two IDL sessions. Session 1:
Now you open a second session and type the same: Session 2:
back to session 1 you fill up the variable: Session 1:
Now on session 2 magically the contents of the variable has changed as well:
And that is what I need. I want to have access on 80 parallel sessions to the same big array so that all the sessions can work on the same data. |
|
in fact |
Ahh, nice, I think we need both. |
Hi @alaingdl just wanted to know if there is any progress in this issue or any workaround? |
FYI, my student @BoudjemaaS just started to have a look on that. Not obvious for us if we are able to do it, but we will try hard, and @Saynkas will help when he will have clarify #1311 😄 |
Thank you for that information!
Good luck!
… On 31. May 2023, at 10:28, Alain ***@***.***> wrote:
FYI, my student @BoudjemaaS <https://github.com/BoudjemaaS> just started to have a look on that. Not obvious for us if we are able to do it, but we will try hard, and @Saynkas <https://github.com/Saynkas> will help when he will have clarify #1311 <#1311> 😄
—
Reply to this email directly, view it on GitHub <#860 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOC5K6AX2P6HWRGFYNMOMELXI5BRXANCNFSM4U6QOFOA>.
You are receiving this because you authored the thread.
|
Hi @alaingdl did you have any progress on this issue? |
Unfortunately no progress. With my student, we did succeed to play with the pure C side of the code without too much problem, but we failed to incorporate it into GDL. It would be great that some experts in C/c++ like @GillesDuvert @slayoo and other give few times on this topic. Not sure CALL_EXTERNAL could be useful on such topic. Any input welcome. |
At first glance this is 'just' having the variable pointer (every GDL variable have such pointer) point to a shared memory region, allocated by mmap() instead of the default allocator. |
Thanks for having a look.
I would have so many ideas on how to accelerate all my processings using all the CPUs that I have on my machine if that shmmap would work.
Anything that I could help with this issue?
Unfortuantely I am not a good C programmer.
… On 23. Nov 2023, at 10:59, Giloo ***@***.***> wrote:
At first glance this is 'just' having the variable pointer (every GDL variable have such pointer) point to a shared memory region, allocated by mmap() instead of the default allocator.
As it goes deep in GDL, and seen the numerous comments in IDL's help page for shmmap, and the need to support all the related functions, it is not 'just' adding a new allocator for variables. But I'm certain it can be managed.
—
Reply to this email directly, view it on GitHub <#860 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOC5K6HWSOJHY3BMQTUQMU3YF5JFRAVCNFSM4U6QOFOKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSGQ2DQNJXGM4Q>.
You are receiving this because you authored the thread.
|
@brandy125 & @GillesDuvert based on https://gist.github.com/drmalex07/5b72ecb243ea1f5b4fec37a6073d9d23 For sure I will need help to go further. I can send the few files by email (send one to me) ... |
As I said in #1677 (comment) I'll do that soon. @alaingdl your code changes are welcome. |
Hi Alain,
if you have something to test please send me.
Chris
… On 24. Nov 2023, at 15:37, Alain ***@***.***> wrote:
based on https://gist.github.com/drmalex07/5b72ecb243ea1f5b4fec37a6073d9d23
I do have a very preliminary working code in GDL, the 2 sessions do exchange memory and I can change the values ...
For sure I will need help to go further. I can send the few files by email ...
—
Reply to this email directly, view it on GitHub <#860 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOC5K6BO4PXQ6XKJOUEW3A3YGDSNNAVCNFSM4U6QOFOKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBSGU4TQOBTGUZA>.
You are receiving this because you authored the thread.
|
Are there any plans to implement shmmap and shmvar in GDL?
The text was updated successfully, but these errors were encountered: