Added an option to count detection upon the center point of the bounding box crossing the line counter #735

revtheundead · 2024-01-16T23:09:38Z

Description

Added an option to count detection upon the center point of the bounding box crossing the line counter.

Added an option to LineZone class specifying the condition which determines whether a detection has crossed the line counter or not.

Additionally made it so that the line counters check whether if the corners (or optionally the center point) of a detection's bounding box are in the line counter's coordinate ranges. This way, line counters count only the detections that have precisely crossed the bounds that are drawn without failing and counting targets that have crossed invisible extensions of the lines. To this end, a new class function is_in_line_range(self, point: Point) that checks whether if a given point is within the coordinate ranges of a line counter is added to class LineZone.

This change was motivated by a use case in a personal project in which a box made from line counters (LineZones) is used to count not only the current count of the targets in the box like in PolygonZone but also to keep tabs on the total the count of targets.

Being able to customize the counting condition of line counters was also seen to have been requested in issue #87 by user @falkaabi.

Extra

An additional function cross_product() was added to the Vector class. This takes the exact same arguments as is_in() function (the function signature is cross_product(self, point: Point) -> float) but returns the actual cross product of the vector with the given point instead of boolean.

This change was made as a small quality of life improvement and a comment was added to better interpret the cross product results.

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

A small rectangle made from 4 separate line counters (LineZones) was formed in the middle of the webcam view. "Bicycle" target class (YOLOv8) was selected to filter detections. A photo of a bicycle was shown to the webcam and the existing and new modes of operation in the line counters were tested.

With the changes made, line counters no longer updated "in count" or "out count" when any of the corners (or center point) of the detection's bounding box fell out of the X and Y ranges of the line counters. Essentially, an update to "in count" or "out count" was made only when every corner (or the center point) of a detection's bounding box was precisely within the coordinate ranges (startx - endx and starty - endy) of the line counter.

Any specific deployment considerations

A new argument was added to LineZone class called count_condition. This argument can take the values whole_crossed or center_point_crossed which determines what condition should be satisfied in order for a detection to be considered as having "crossed" the line counter.

count_condition is set to whole_crossed by default. This is the previous behaviour of the class prior to the changes with the exception of also checking the coordinate of every bounding box anchor against the line counter's ranges.

When count_condition is set to center_point_crossed, the center point of the bounding box is calculated and the detection is said to have crossed the line counter only if the center point has a negative product with its previous state and is within the line counter's coordinate ranges.

Docs

Docs updated? What were the changes:
New argument count_condition added to class LineZone.
New class method is_in_line_range() added to class LineZone.
New function cross_product() added to class Vector.

…ing box crossing the line counter. Added an option to LineCounter class specifying the condition which determines whether a detection has crossed the line counter or not. Additionally made it so that the line counters check whether if the corners (or optionally the center point) of a detection's bounding box are in the line counter's coordinate ranges. This way, line counters count only the detections that have precisely crossed the bounds that are drawn without failing and counting targets that have crossed invisible extensions of the lines.

CLAassistant · 2024-01-16T23:09:44Z

All committers have signed the CLA.

SkalskiP · 2024-01-17T10:34:22Z

Hi, @revtheundead 👋🏻 ! Thanks a lot for your interest in supervision. I was thinking about adding other triggering strategies for quite some time. I'll try to review this code as fast as possible.

SkalskiP · 2024-01-17T10:39:20Z

@revtheundead, would you be willing to implement changes some changes? Looks like I will have a few comments. Nothing huge. ;)

revtheundead · 2024-01-17T11:16:48Z

@revtheundead, would you be willing to implement changes some changes? Looks like I will have a few comments. Nothing huge. ;)

Of course :) I'll be on them as soon as I'm available.

SkalskiP · 2024-01-17T17:05:03Z

supervision/geometry/core.py

@@ -51,6 +51,16 @@ def is_in(self, point: Point) -> bool:
        ) * (v2.end.x - v2.start.x)
        return cross_product < 0

+    def cross_product(self, point: Point) -> float:


There are a few things I would like to address.

We generally require function/methods arguments and results to be documented in docstring. This docstring is later used to generate our documentation website automatically.

Secondly, we already have is_in method, responsible for very similar calculations. I don't like the is_in name. cross_product is a lot better. I'm happy that you made this change:

# old triggers = [self.vector.is_in(point=anchor) for anchor in anchors] # new triggers = [ self.vector.cross_product(point=anchor) < 0 for anchor in anchors ]

Please remove the old is_in method, and adjust test_vector_is_in to test your new method.

I made the requested adjustments. Please check them if any further modifications is needed.

SkalskiP · 2024-01-17T17:07:39Z

supervision/detection/line_counter.py

-            if tracker_id is None:
-                continue
+        if self.count_condition == "whole_crossed":
+            for i, (xyxy, _, confidence, class_id, tracker_id) in enumerate(detections):


That line was changed recently. Please reflect that change in your new implementation.

Changed this line with the new commit.

SkalskiP · 2024-01-17T18:58:58Z

supervision/detection/line_counter.py

+                "Argument count_condition must be 'whole_crossed' or 'center_point_crossed'"
+            )
+
+    def is_point_in_line_range(self, point: Point) -> bool:


Could you share a bit more about this change? Does it work with diagonal lines? Adding a behavior like the one shown in the picture has been on my mind for some time.

The idea is to check whether the bounding box points of interest ( in this case, the bottom center) are between the dashed lines.

The given function works for the above case of a diagonal line. In the case of purely vertical and horizontal lines; Xstart, Xend and Ystart, Yend respectively are the same. So in those cases only the coordinate range that is meaningful to check is being asserted. In all other cases (such as the diagonal line in the image) both coordinate ranges should hold for the point of interest to be considered "in range".

However, it should be noted that in the current way the function works, the dashed lines are parallel to the axes instead of being perpendicular to the line as shown above. So in reality the area considered as "in range" of the line counter's coordinates is much smaller. This could potentially lead to some bugs when there are multiple triggering anchors that are far from each other, in which case the line counter won't count the object as having crossed in or out.

Thinking of a fix for that in the mean time :)

SkalskiP · 2024-01-17T18:59:37Z

supervision/detection/line_counter.py

+            return self.in_count, self.out_count
+
+        elif self.count_condition == "center_point_crossed":
+            for i, (xyxy, _, confidence, class_id, tracker_id) in enumerate(detections):


That line was changed recently. Please reflect that change in your new implementation.

That check is no longer necessary so it was removed with the new commit.

SkalskiP · 2024-01-17T19:00:45Z

supervision/detection/line_counter.py

-                self.out_count += 1
-                crossed_out[i] = True
+                # Update the tracker state and check for crossing
+                if previous_state * current_state < 0 and self.is_point_in_line_range(


Please do not include comments in code :)

Oh, you are right. I'll try not to leave any more comments :)

SkalskiP · 2024-01-17T19:05:57Z

supervision/detection/line_counter.py

+                x1, y1, x2, y2 = xyxy
+
+                # Calculate the center point of the box
+                center_point = Point(x=(x1 + x2) / 2, y=(y1 + y2) / 2)


Let's not limit ourselves to just the center of the box. Let's make it more general. Also, you can use get_anchors_coordinates to calculate coordinates of specific bounding box points. (already tested with multiple unit tests)

I have used the function you mentioned in the new commit.

SkalskiP · 2024-01-17T19:14:09Z

supervision/detection/line_counter.py

@@ -26,16 +26,61 @@ class LineZone:
            to outside.
    """

-    def __init__(self, start: Point, end: Point):
+    def __init__(self, start: Point, end: Point, count_condition="whole_crossed"):


Here's an idea. Let's replace count_condition with triggering_anchors: List[sv.Position]. In this way, we can provide our users with full flexibility.

Let me know if you need more explanation. :)

I believe I implemented just what you needed. Please check so we can discuss further if need be.

…function

…h finding anchors

…on-to-line-counter' into add-center-point-crossed-condition-to-line-counter

This reverts commit 9fb97e3.

This reverts commit 353865e.

revtheundead · 2024-01-18T10:52:11Z

Hey @SkalskiP, I made some small mistakes and reverted them. Sorry for the confusion but it should all be good to go now. Let me know if you have any questions.

SkalskiP · 2024-01-18T11:31:44Z

@revtheundead, please don't work on this branch now. I'm testing a few things around is_point_in_line_range 🙏🏻

…s added.

SkalskiP · 2024-01-18T14:05:58Z

@revtheundead, it looks like I managed to successfully implement the logic to filter detections located in the region of interest. I just need to plug it into your code, and we should be good.

revtheundead · 2024-01-18T14:27:28Z

@SkalskiP, thats great! Hope I was of some help. I hadn't noticed that something was wrong with the logic. Would you mind sharing what was missing and how you fixed it? Asking out of curiosity :)

SkalskiP · 2024-01-18T15:12:19Z

@revtheundead I just described it on Twitter: https://x.com/skalskip92/status/1747998962429186111?s=20! Take a look!

Hope I was of some help.

Of course, you were! I'll keep everything else. Just replace your is_point_in_line_range with new is_point_in_limits.

revtheundead · 2024-01-18T15:55:12Z

@revtheundead I just described it on Twitter: https://x.com/skalskip92/status/1747998962429186111?s=20! Take a look!

Hope I was of some help.

Of course, you were! I'll keep everything else. Just replace your is_point_in_line_range with new is_point_in_limits.

I suppose you made the necessary changes and it looks good to me. Let me know if you need further modifications :)

…e-counter

SkalskiP · 2024-01-18T16:21:40Z

@revtheundead, thanks a lot for your help! I believe we made LineZone a lot better today. This PR brings in a new, strong mechanism and solves a bug identified months ago.

Only detections ACTUALLY crossing the line are taken into consideration. If the object passes the line from the side, it is discarded.

vehicle-counting-result-left-lane-4-anchors-optimized.mp4

vehicle-counting-result-right-lane-4-anchors-optimized.mp4

You can select the list of bounding box anchors that need to cross the line for the object to be counted as crossing the line.

vehicle-counting-result-left-lane-1-anchor-bottom-center-optimized.mp4

revtheundead · 2024-01-18T16:28:23Z

@SkalskiP, its been my pleasure! Seeing the line counters work this efficiently makes me think that the effort really paid off. It would be nice to contribute more when I can.

revtheundead · 2024-01-18T16:39:30Z

@SkalskiP I hope you don't mind me mentioning this improvement and our collaboration on LinkedIn. With credits of course :)

SkalskiP · 2024-01-18T16:44:14Z

@revtheundead, not at all! I always write a post related to supervision releases and tag all contributors. supervision-0.18.0 is probably coming next week, so I'll tag you 100%.

revtheundead · 2024-01-18T16:45:39Z

@revtheundead, not at all! I always write a post related to supervision releases and tag all contributors. supervision-0.18.0 is probably coming next week, so I'll tag you 100%.

@SkalskiP Awesome! I'd be honored.

revtheundead added 2 commits January 17, 2024 01:24

Change cross_product() return type from int to float

602676b

fix(pre_commit): 🎨 auto format pre-commit hooks

47eed72

SkalskiP requested changes Jan 17, 2024

View reviewed changes

revtheundead and others added 10 commits January 18, 2024 01:21

Increased line counter flexibility and replaced vector cross product …

1f64098

…function

Resolved merge conflict

2367fca

Fixed the is_in_line_range() function logic and fixed a minor bug wit…

ae758a6

…h finding anchors

Fixing cross_product tests.

da4ee62

Fixed cross_product() funtion test

353865e

Formatting updates.

008b0bd

Merge remote-tracking branch 'origin/add-center-point-crossed-conditi…

3a608ee

…on-to-line-counter' into add-center-point-crossed-condition-to-line-counter

Implemented cross_product() with numpy

9fb97e3

Revert "Implemented cross_product() with numpy"

ec7baf3

This reverts commit 9fb97e3.

Revert "Fixed cross_product() funtion test"

dca7fb4

This reverts commit 353865e.

SkalskiP added 2 commits January 18, 2024 12:46

Adding ability to calculate vector magnitude.

616fc96

calculate_region_of_interest_limits and is_point_in_limits method…

5f04836

…s added.

ready for end-to-end tests.

f8ee3a4

SkalskiP and others added 2 commits January 18, 2024 17:06

Merge branch 'develop' into add-center-point-crossed-condition-to-lin…

cda5016

…e-counter

fix(pre_commit): 🎨 auto format pre-commit hooks

6128576

SkalskiP merged commit c04edbc into roboflow:develop Jan 18, 2024
8 checks passed

revtheundead deleted the add-center-point-crossed-condition-to-line-counter branch January 18, 2024 19:05

This was referenced Jan 19, 2024

Separate line and zone counters for separate classes #87

Closed

supervision-0.18.0 release #764

Merged

SkalskiP mentioned this pull request Feb 2, 2024

[LineZone] - flip in/out line crossing directions #838

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added an option to count detection upon the center point of the bounding box crossing the line counter #735

Added an option to count detection upon the center point of the bounding box crossing the line counter #735

Added an option to count detection upon the center point of the bounding box crossing the line counter #735

Added an option to count detection upon the center point of the bounding box crossing the line counter #735

Conversation

Description