[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix smoother removing tracking_id from detections #944

Merged

Conversation

LinasKo
Copy link
Collaborator
@LinasKo LinasKo commented Feb 25, 2024

Addresses issue #928 matching implementation in ByteTrack, but sidestepping the underlying cause outlined in #943.

Description

Smoother takes as input Detections with a defined tracker_id, but when no detections are left after smoothing, it would return an object where tracker_id is None.

When accessing such detections, for example, using zip(detections.class_id, detections.tracker_id), a NoneType is not iterable error would be raised.

Similar to implementation of ByteTrack.update_with_detections() does it, the tracker_id is added explicitly in cases where it's not present already (in fact - only when no detections are left).

Fixes issue: #928
Related Issue: #943

List any dependencies that are required for this change.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How has this change been tested, please provide a testcase or example of how you tested the change?

Input Video:
https://github.com/roboflow/supervision/assets/6500785/d61b7e9d-2992-46d0-9cd1-6a02388dbe72

Test Code:

Test code
import supervision as sv
from pathlib import Path
from ultralytics import YOLO

VIDEO_FILE = Path('data', 'test-traffic.mp4')
VIDEO_OUT = Path('data', 'test-traffic-annotated.mp4')


# Only a regression test. PIL will never be used to open video files.
trace_annotator = sv.LabelAnnotator()

model_det = YOLO("yolov8n")
video_info = sv.VideoInfo.from_video_path(video_path=str(VIDEO_FILE))
frames_generator = sv.get_video_frames_generator(source_path=str(VIDEO_FILE))
tracker = sv.ByteTrack()
smoother = sv.DetectionsSmoother()

with sv.VideoSink(target_path=str(VIDEO_OUT), video_info=video_info) as sink:
    for frame in frames_generator:
        result = model_det(frame)[0]
        detections = sv.Detections.from_ultralytics(result)
        detections = tracker.update_with_detections(detections)
        smoother.update_with_detections(detections)
        detections = smoother.get_smoothed_detections()

        # print(detections.tracker_id, detections.class_id)
        labels = [
            f"#{tracker_id} {class_id}"
            for tracker_id, class_id
            in zip(detections.tracker_id, detections.class_id)
        ]

        annotated_frame = trace_annotator.annotate(
            scene=frame.copy(),
            detections=detections,
            labels=labels
        )
        sink.write_frame(frame=annotated_frame)

Any specific deployment considerations

None.

Docs

No updates.

* Addresses issue roboflow#928 matching implementation in ByteTrack,
  but sidestepping the underlying cause outlined in roboflow#943.
@LinasKo
Copy link
Collaborator Author
LinasKo commented Feb 25, 2024

Ready for review.

@@ -115,4 +115,8 @@ def get_smoothed_detections(self) -> Detections:
if track is not None:
tracked_detections.append(track)

return Detections.merge(tracked_detections)
detections = Detections.merge(tracked_detections)
if len(tracked_detections) == 0:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if len(tracked_detections) == 0:
if len(detections) == 0:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @LinasKo 👋🏻 ! Great find! That's probably the only change I'd make.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 👍

Also verified the __len__ implementation just in case. Seems to be alright - can't imagine changes to Detections.empty that would break this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm considering modifying the Detections.empty API to include arguments such as with_confidence, with_tracker_id, etc. So that in the future, in a similar situation, we would only need to do Detections.empty(with_tracker_id = True).

Copy link
Collaborator Author
@LinasKo LinasKo Feb 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with that (#943, Option 4) is that intermediary functions would also need with_confidence and with_tracker_id, or you need to forbid them from ever accepting zero-length iterables.

We're actually at the best place to show this.

If nothing is detected, detections = Detections.merge(tracked_detections) takes []. It then tries to call Detections.empty - but what params should it set?..

@SkalskiP SkalskiP merged commit 13cb923 into roboflow:develop Feb 28, 2024
8 checks passed
@LinasKo LinasKo deleted the fix/smoother-removes-detections-tracker-id branch April 11, 2024 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

[Smoother] - tracker_id is None when there is no detection for an extended period
2 participants