[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add transfer time settings for routing (with LB) #114

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Add transfer time settings for routing (with LB) #114

wants to merge 7 commits into from

Conversation

pablohoch
Copy link
Member
@pablohoch pablohoch commented Jun 19, 2024 •
edited
Loading

Same as #113, but with adjusted lower bounds.

Routing performance seems to be very similar:

Benchmark results from nigiri-benchmark with options --start_mode station --dest_mode station -m 0 -e false -l false -n 5000 + transfer time settings, current DELFI GTFS timetable

Default transfer times

#113 (default LB):

--- total_time --- (n = 5000)
  10%: (t_total: 00000.385s, t_exec: 00000.360s, intvl_ext: 00, intvl_size: 00001h, #jrny:  2)
  20%: (t_total:     0.613s, t_exec:     0.559s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  30%: (t_total:     0.787s, t_exec:     0.751s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  40%: (t_total:     0.943s, t_exec:     0.902s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  50%: (t_total:     1.097s, t_exec:     1.064s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  60%: (t_total:     1.273s, t_exec:     1.240s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  70%: (t_total:     1.461s, t_exec:     1.436s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  80%: (t_total:     1.705s, t_exec:     1.674s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  90%: (t_total:     2.053s, t_exec:     2.007s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  99%: (t_total:     2.961s, t_exec:     2.923s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
99.9%: (t_total:     3.847s, t_exec:     3.807s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  max: (t_total:     4.045s, t_exec:     4.014s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
----------------------------------

#114 (adjusted LB):

--- total_time --- (n = 5000)
  10%: (t_total: 00000.402s, t_exec: 00000.365s, intvl_ext: 00, intvl_size: 00001h, #jrny:  3)
  20%: (t_total:     0.616s, t_exec:     0.581s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  30%: (t_total:     0.791s, t_exec:     0.757s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  40%: (t_total:     0.949s, t_exec:     0.909s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  50%: (t_total:     1.102s, t_exec:     1.062s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  60%: (t_total:     1.266s, t_exec:     1.221s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  70%: (t_total:     1.456s, t_exec:     1.402s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  80%: (t_total:     1.656s, t_exec:     1.602s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  90%: (t_total:     1.996s, t_exec:     1.951s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  99%: (t_total:     2.891s, t_exec:     2.839s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
99.9%: (t_total:     4.019s, t_exec:     3.967s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  max: (t_total:     5.109s, t_exec:     5.065s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
----------------------------------

Min transfer time 10 minutes

#113 (default LB):

--- total_time --- (n = 5000)
  10%: (t_total: 00000.362s, t_exec: 00000.331s, intvl_ext: 00, intvl_size: 00001h, #jrny:  1)
  20%: (t_total:     0.576s, t_exec:     0.515s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  30%: (t_total:     0.752s, t_exec:     0.727s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  40%: (t_total:     0.891s, t_exec:     0.851s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  50%: (t_total:     1.030s, t_exec:     0.977s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  60%: (t_total:     1.166s, t_exec:     1.132s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  70%: (t_total:     1.329s, t_exec:     1.304s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  80%: (t_total:     1.530s, t_exec:     1.505s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  90%: (t_total:     1.838s, t_exec:     1.785s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  99%: (t_total:     2.658s, t_exec:     2.628s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
99.9%: (t_total:     3.242s, t_exec:     3.207s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  max: (t_total:     3.947s, t_exec:     3.913s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
----------------------------------

#114 (adjusted LB):

--- total_time --- (n = 5000)
  10%: (t_total: 00000.362s, t_exec: 00000.302s, intvl_ext: 00, intvl_size: 00001h, #jrny:  2)
  20%: (t_total:     0.571s, t_exec:     0.525s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  30%: (t_total:     0.737s, t_exec:     0.694s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  40%: (t_total:     0.878s, t_exec:     0.838s, intvl_ext:  0, intvl_size:     1h, #jrny:  4)
  50%: (t_total:     1.007s, t_exec:     0.946s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  60%: (t_total:     1.155s, t_exec:     1.116s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  70%: (t_total:     1.334s, t_exec:     1.273s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  80%: (t_total:     1.551s, t_exec:     1.500s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  90%: (t_total:     1.840s, t_exec:     1.801s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  99%: (t_total:     2.608s, t_exec:     2.514s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
99.9%: (t_total:     3.885s, t_exec:     3.833s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  max: (t_total:     3.947s, t_exec:     3.885s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
----------------------------------

Transfer time factor 2.0x

#113 (default LB):

--- total_time --- (n = 5000)
  10%: (t_total: 00000.368s, t_exec: 00000.342s, intvl_ext: 00, intvl_size: 00001h, #jrny:  1)
  20%: (t_total:     0.596s, t_exec:     0.564s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  30%: (t_total:     0.775s, t_exec:     0.741s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  40%: (t_total:     0.931s, t_exec:     0.853s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  50%: (t_total:     1.079s, t_exec:     1.047s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  60%: (t_total:     1.240s, t_exec:     1.198s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  70%: (t_total:     1.427s, t_exec:     1.379s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  80%: (t_total:     1.649s, t_exec:     1.617s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  90%: (t_total:     1.981s, t_exec:     1.940s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  99%: (t_total:     2.901s, t_exec:     2.872s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
99.9%: (t_total:     3.885s, t_exec:     3.860s, intvl_ext:  0, intvl_size:     1h, #jrny:  3)
  max: (t_total:     4.838s, t_exec:     4.800s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
----------------------------------

#114 (adjusted LB):

--- total_time --- (n = 5000)
  10%: (t_total: 00000.405s, t_exec: 00000.369s, intvl_ext: 00, intvl_size: 00001h, #jrny:  1)
  20%: (t_total:     0.629s, t_exec:     0.573s, intvl_ext:  0, intvl_size:     1h, #jrny:  5)
  30%: (t_total:     0.793s, t_exec:     0.724s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  40%: (t_total:     0.938s, t_exec:     0.888s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  50%: (t_total:     1.088s, t_exec:     1.044s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  60%: (t_total:     1.249s, t_exec:     1.208s, intvl_ext:  0, intvl_size:     1h, #jrny:  1)
  70%: (t_total:     1.425s, t_exec:     1.377s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  80%: (t_total:     1.660s, t_exec:     1.609s, intvl_ext:  0, intvl_size:     1h, #jrny:  2)
  90%: (t_total:     2.014s, t_exec:     1.967s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  99%: (t_total:     2.878s, t_exec:     2.836s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
99.9%: (t_total:     4.021s, t_exec:     3.980s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
  max: (t_total:     4.300s, t_exec:     4.269s, intvl_ext:  0, intvl_size:     1h, #jrny:  0)
----------------------------------

This was referenced Jun 19, 2024
Open
Draft
@felixguendling
Copy link
Member

In case the factor is negative, 100% correctness is probably only possible if LBs are adjusted too. Otherwise the LB is not correct and there for journeys will be dominated that would have a chance to dominate.

@felixguendling
Copy link
Member

Is the measurement for "Default transfer times" from the master branch or from this branch?

Maybe we should also test with the transitous dataset (at least for Europe)?

@mority can you maybe help out with your current default benchmarking configuration? :)

@pablohoch
Copy link
Member Author

In case the factor is negative, 100% correctness is probably only possible if LBs are adjusted too. Otherwise the LB is not correct and there for journeys will be dominated that would have a chance to dominate.

True. Do we want to support factors < 1.0?

Is the measurement for "Default transfer times" from the master branch or from this branch?

All the measurements are from the versions with transfer time settings - first the version without LBs (#113), then the version with LBs. I haven't measured the master branch yet.

@mority
Copy link
Contributor
mority commented Jun 20, 2024

Is the measurement for "Default transfer times" from the master branch or from this branch?

Maybe we should also test with the transitous dataset (at least for Europe)?

@mority can you maybe help out with your current default benchmarking configuration? :)

I usually go with the defaults:

  • interval_size: 1h
  • start_mode, dest_mode: intermodal
  • intermodal_start_mode, intermodal_dest_mode: walk
  • max_transfers: 15
  • min_connection_count: 3
  • extend_interval_earlier, extend_interval_later: true

For the transitous dataset, you can use -b europe to restrict the start and destination locations to a bounding box of Europe. I often start with 100 queries for an initial comparison and run greater batch sizes if an optimization looks promising.

I use the randomized seed that the baseline run prints as input -s <seed_value> for the consecutive runs. This ensures that all benchmarks are run on the same queries. I guess, for larger batch sizes it does not matter since the law of large numbers would take care of convergence to the mean. But this way, we can compare smaller batches which allows for shorter benchmark runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@pablohoch @felixguendling @mority