[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Question] Impact of WorkflowRegistry storing all definitions in memory. #1207

Open
Julian-Robinson opened this issue Sep 13, 2023 · 1 comment
Open

[Question] Impact of WorkflowRegistry storing all definitions in memory. #1207

Julian-Robinson opened this issue Sep 13, 2023 · 1 comment

Comments

@Julian-Robinson
Copy link

Hi team

We're evaluating whether Workflow-Core is suitable option for us as a workflow engine to drive an end user automation feature to allow our users to create their own simple automation steps (eg, when x happens, then do y then z). Our approach so far is that when a user sets up a workflow, we create and register a workflow definition to represent it and it seems to be working. But with this model, there will likely be many thousand Workflow Definitions as users set up their own automation rules.

One thing we've noticed is in the WorkflowRegistry implementation (below) it loads all definitions on startup and stores them in a concurrent dictionary in memory. We're wondering whether there may be a ceiling we hit here if we have a lot of definitions.

workflow-core/src/WorkflowCore/Services/WorkflowRegistry.cs

Lines 11 to 35 in ab97ce9

public class WorkflowRegistry : IWorkflowRegistry
{
private readonly IServiceProvider _serviceProvider;
private readonly ConcurrentDictionary<string, WorkflowDefinition> _registry = new ConcurrentDictionary<string, WorkflowDefinition>();
private readonly ConcurrentDictionary<string, WorkflowDefinition> _lastestVersion = new ConcurrentDictionary<string, WorkflowDefinition>();
public WorkflowRegistry(IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
}
public WorkflowDefinition GetDefinition(string workflowId, int? version = null)
{
if (version.HasValue)
{
if (!_registry.ContainsKey($"{workflowId}-{version}"))
return default;
return _registry[$"{workflowId}-{version}"];
}
else
{
if (!_lastestVersion.ContainsKey(workflowId))
return default;
return _lastestVersion[workflowId];
}

So I guess my questions are:

  • What kind of scale in terms of a rough number of definitions, would you expect Workflow core to support?
  • Is our approach something that you would consider suitable for Workflow-Core?

Thanks, and we really appreciate your time looking over this.
@danielgerlag
Copy link
Owner

You could create an alternative implementation of IWorkflowRegistry that persists to a datastore and swap it out with the in-memory one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@danielgerlag @Julian-Robinson