[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import is really really slow #109

Closed
AntoninHL opened this issue Jul 2, 2019 · 7 comments
Closed

Import is really really slow #109

AntoninHL opened this issue Jul 2, 2019 · 7 comments
Assignees
@richard-julien
Labels
feature use for describing a new feature to develop solved use to identify issue that has been solved (must be linked to the solving PR)
Milestone
Release 2.1.0

Comments

@AntoninHL
Copy link

I started MISP import since 3 hours...
Only 268 observables, 1 report and 5 entities...
馃槸
Is it normal?
@SamuelHassine
Copy link
Member

@Wesh: depending on your hardware but we are aware of that. Our next priority is to provide statistics and status for all connectors and progress of integration and implement parallel insertions. We will keep you informed on this issue on the progresses we made on it in the next days.

@AntoninHL
Copy link
Author

馃憤

@AntoninHL
Copy link
Author

1471 observables in 22 hours...
馃槬

image

@SamuelHassine
Copy link
Member

@Wesh: we found a quick solution to enhance the MISP connector and speed-up the ingestion speed.

@AntoninHL
Copy link
Author

@SamuelHassine Good news!
I can't wait to try it!
馃槈

@SamuelHassine
Copy link
Member

@Wesh: you can follow the setup of this feature here in this issue.

@SamuelHassine SamuelHassine self-assigned this Jul 3, 2019
@SamuelHassine SamuelHassine added the feature use for describing a new feature to develop label Jul 3, 2019
@SamuelHassine SamuelHassine added this to the Release 1.0.2 milestone Jul 3, 2019
@AntoninHL
Copy link
Author

@SamuelHassine
A very strange thing: for some reasons, I must restart my server.
And after restart, MISP import restart at the begining... 馃槸
So, the last 28 hours passed to import 1450 observables are lost...

SamuelHassine pushed a commit that referenced this issue Jul 5, 2019
#116 No more install in Docker entrypoint, #114 Correct volumes config,
8854c25 
#109 Use workers to import, #113 Fix, #85 More Observables types
@SamuelHassine SamuelHassine added the solved use to identify issue that has been solved (must be linked to the solving PR) label Jul 7, 2019
@SamuelHassine SamuelHassine reopened this Nov 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@richard-julien richard-julien

Labels
feature use for describing a new feature to develop solved use to identify issue that has been solved (must be linked to the solving PR)
Projects
None yet
Milestone
Release 2.1.0
Development

No branches or pull requests
3 participants
@AntoninHL @richard-julien @SamuelHassine