-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Prediction Smoothing #696
Conversation
supervision/detection/smoother.py
Outdated
> _On the left are the model's raw predictions, | ||
> on the right is the output of Smoother._ | ||
|
||
## Example Usage: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we show a more general example? I assume Smoother
can work with any object detection model.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, what do you have in mind? I figured person detection with COCO using one of the videos from assets
was about as general as it gets to demonstrate the functionality but we can change it to something else.
Good candidate videos to communicate the functionality would have
- A small number of objects moving fairly subtly (otherwise it's really distracting trying to understand what you're supposed to be comparing side by side)
- Prediction jitter coming from the model
For example, while it works with vehicles-2.mp4, it's not a good example video because there's too much going on and the things are moving too fast to be able to grok what's going on at the detection by detection level.
@yeldarby, please let me know once the PR is ready for the next review round. |
…pervision into prediction-smoothing
…pervision into prediction-smoothing
PR is updated in response to your comments here & @capjamesg's comments on Slack. |
@yeldarby I started the second round of review, but I got confused and decided to ask here:
All around, the whole logic can get a lot easier if we drop |
I'm using it in some client code to create animated Also planning to use it to let people choose if we should wait to display a detection until it's been seen some number of times so that if something is detected in a frame or two but not again it doesn't flicker onto the screen (but haven't implemented that here just yet).
Good call; if we keep it here I'll document it.
The entry delay will be related to this (though that's easier since you should be able to just take the length of the non- But yeah I think this probably makes a bit more structural sense at the |
Updated to remove |
for track_id in self.tracks: | ||
self.tracks[track_id] = deque(self.tracks[track_id], maxlen=length) | ||
|
||
def tracker_length(self, tracker_id: int) -> int: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like this method is unused. We don't expose it in docs. Can we just remove it?
|
||
return self.get_smoothed_detections() | ||
|
||
def get_track(self, track_id: int) -> Optional[dict]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we have an incorrect return type here. Shouldn't it be Optional[Detections]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated in my latest commit.
if len(track) == 0: | ||
return None | ||
|
||
ret = track[0] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like you mutate that object every time. Is that intended? Let's create a copy here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated in my latest commit.
self.tracks[tracker_id].append(detections[detection_idx]) | ||
self.track_ends[tracker_id] = self.current_frame | ||
|
||
for track_id in self.tracks: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have no state-clearing mechanism. After some time, most of the tracks we store will be dead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@SkalskiP How would you recommend we clear state?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we have an explicit method DetectionsSmoother.clear()
, or some logic here?
from supervision.detection.core import Detections | ||
|
||
|
||
class Smoother: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think about changing the name to DetectionsSmoother
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated in my latest commit.
@SkalskiP In my latest change, I add logic to purge unused tracker IDs: used_tracker_ids = {}
for detection_idx in range(len(detections)):
tracker_id = detections.tracker_id[detection_idx]
if tracker_id is None:
# skip detections without a tracker id
continue
if self.tracks.get(tracker_id, None) is None:
self.tracks[tracker_id] = deque(maxlen=self.length)
self.tracks[tracker_id].append(detections[detection_idx])
self.track_ends[tracker_id] = self.current_frame
used_tracker_ids[tracker_id] = True
for track_id in list(self.tracks.keys()):
if track_id not in used_tracker_ids:
del self.tracks[track_id]
del self.track_ends[track_id] Here is an example of the smoothing running on a video: container_output_1.1.mp4 |
Description
Adds a
Smoother
class which averages the last several predictions to decrease noise (like wobbly or flickering boxes).Before & after smoothing:
smoothed-grocery-example-480.mp4
Type of change
How has this change been tested, please provide a testcase or example of how you tested the change?
Installed locally & tested on several videos/models with
InferencePipeline
.Any specific deployment considerations
No.
Docs