PHRF 2009 Madaket Hotlaps Discussion Thread

[Jane Fossett]

This is a thread to facilitate an ongoing discussion of PHRF handicapping and the Madaket Hotlaps Racelinewe're using to collect data.

The entire idea of PHRF is to produce a valid and reliable handicapping system that allows different classes of boats to race against each other under the same conditions.
I'll append a series of notes and updates to this thread and add links to more material as well.

[Tig Spijkers]

Hello Jane,

It is as you said under the same conditions.
The conditions change with every course and wind settings.
Many boats we see in open races have different characteristics.
One race course may be more favorable for a boat that can sail a small angle upwind, but is other wise slow.

I can even imagine creating a course with specific wind settings that would make the VOJ70 look like an old goose compaired to the ACA.

I have been noticing these differences on many occassions now and I know that other RD's have been too.

To stay with these two boats there are charateristics that cannot be denied.
A long straight course of approx. 20 minutes with preferable winds that keep the windangle above 40 degrees an ACA will finish anywhere between 3 to 5 minutes later than the VOJ70.
On shorter courses they seem to match very well:
http://slsailing.org/forum/viewtopic.php?t=3155

What are your experiences on this?

[Jane Fossett]

Hello Jane,

It is as you said under the same conditions.
The conditions change with every course and wind settings.
Many boats we see in open races have different characteristics.
One race course may be more favorable for a boat that can sail a small angle upwind, but is other wise slow.
I can even imagine creating a course with specific wind settings that would make the VOJ70 look like an old goose compaired to the ACA.
I have been noticing these differences on many occassions now and I know that other RD's have been too.
To stay with these two boats there are charateristics that cannot be denied.
A long straight course of approx. 20 minutes with preferable winds that keep the windangle above 40 degrees an ACA will finish anywhere between 3 to 5 minutes later than the VOJ70.
On shorter courses they seem to match very well:
http://slsailing.org/forum/viewtopic.php?t=3155
What are your experiences on this?

Hi Tig!! Great question, since the real issue with PHRF is how 'valid and reliable' the handicapping is (i.e., not only the rank order of boats from fastest to slowest, but also the parametric weighting of each of the handicap factors) on any race course, not just the "PHRF test track." If PHRF results only worked for one race course, that might mean the results were "reliable" and "reproducible" there, but that handicap wouldn't be "valid," since in this case, valid means an authentic index of the relative race performance of a boat under most conditions.
In order to answer the latter question, we kept switching racecourses as soon as we had enough data. I've attached six of the courses we looked at.
We spent a lot of time thinking about this and planning different courses... Vin Mariani and I spent more than three weeks, in fact, trying to match our list of criteria with a racecourse configuration close to NYC. We finally ended up with the "NYC hot laps 2008" course. When we developed some doubts about that course based on a single boat's quite minor but nevertheless consistent and inexplicable deviations, M1sha Dallin and I came up with a new course and we even posted it for a couple weeks on org to get feedback before using it. (how often does that happen?)
If this approach sounds sort of nuts, grin... good!!! The entire thing about handicapping is making it believable. Going overboard with repeated measurements, switching courses, setting up new prospective and retrospective hypotheses... helped make the whole thing believable.
Oh! Getting back to the question you asked, here is the bottom line:
All of the race courses we looked at gave nearly identical results. The rank order and parametric values for the handicaps were remarkably similar. That was part of the evidence our test procedures and handicap scores were both valid and reliable relative indexes of boat performance.
Having said that, I absolutely agree with you that it might be possible to design a course that favored one boat over another, but our results argue that would be a very strange looking course and also strongly suggest a bad boat design with a seriously skewed polar. I'll go back and take a look at the data and the posts and see if anything else shows up worth mentioning.
xxx

[Jane Fossett]

I'm beginning to look at the Madaket Hotlaps data.
This is all a "first pass" look, while we decide and set up the best way to present HL data and make it easily updated.
I'm posting a couple examples here to show sailors what the actual data looks like. I will next set up a few webpages to store all of the actual individual lap scores so sailors can see what the entire fleet is doing and has done.
So far it seems to be working fairly well, although I think we can do a lot more to clean up the data and make things even more convenient for skippers.
If you look at the snapshot from the spreadsheet below, you'll see the line collects and stores the skippers name, the type of boat, the name of the course, and the start/ finish times.
The skipper needs to actually tell the database to store the information, but a sailor can do that as long as several days after using the line.
Once a sailor tells the database to save a lap, it becomes part of the permanent data set.

So far the database has 180 laps run in 21 different boats skippered by 25 sailors. The sailors were Gemma Vuckovic, Francois Jacques, Carmen Foden, joro Aya, Garrick Diesel, Justbent Clarity, Heidi Stiglitz, Angus Moonwall, Dunan Wilder, Chaos Mandelbrot, Cory Copeland, LDeWell Hawker, Oliphant Ming, Julia Ceres, Liv Leigh, Triton Sands, Hal Burnstein, Blackbird Latte, Argus Farman, Vin Mariani, Rodman Mapholisto, Alain Gloster, Jane Fossett, nobuko Criss, and Masahisa Greenwood.

Here are a few observations:
If you look at the data table, you'll see that the name of the course is listed differently by different people. At the present time we are only collecting data on the Madaket PHRF hotlaps course, which can be tedious to write each time. We've changed the entry form to give you a drop-down list showing different courses (currently there is only one listed).
Another possible source of error is the skipper's name. The database actually enters the owner of the boat, not the skipper, so if you have a long list of friends who use your boat, tell them to 'edit' their lap entries before they save them. It's pretty easy to do that, just follow the directions at the line; you don't even have to get out of your boat.
The database records the boat's name (object name field), so don't change it to anything unrecognizable. "Tako 3.3 will never die" is OK, but "I love Mommy" isn't.

I still need to prune the database for false entries, and come up with a systematic way to check for that. If you look below at the yellow arrows, for example, you'll see three entries with lap times that are clearly way beyond a normal distribution range for lap scores. It's likely the individual crashed or made some other error, and entering didn't realize they the data clicked the key to save those scores.

There are only four or five such entries in the entire data set of 180 laps, so don't worry about making mistakes. If you think something went wrong just drop me a note and I'll fix-it online for you. Don't forget, you can always doublecheck your own data set, so you'll be able to tell if it's okay or not.

[Tig Spijkers]

Hello Jane,

Thank you for all the information.

I don't want to sound to silly, but the first time I saw these numbers I was confused.

This should go under FAQ's I suppose

Am I supposed to multiple the lap times with the handicap number?

I know the answer is yes, but I also know many sailors are not aware what to do with the numbers.

[Jane Fossett]

Hello Jane,
Thank you for all the information.
I don't want to sound to silly, but the first time I saw these numbers I was confused.
This should go under FAQ's I suppose
Am I supposed to multiple the lap times with the handicap number?
I know the answer is yes, but I also know many sailors are not aware what to do with the numbers.

Hi!
Sorry I'm posting this stuff piecemeal; I'm sure its confusing.
I'm in the process of setting up new webpages specifically for Madaket PHRF Hotlaps, which should make everything a lot easier.
A discussion of Madaket 2009 hotlaps is here.
You can get the same details by clicking the chart display at the madaket raceline.
That gives you a copy of the chart and details on how to use the line and web display board. It's actually pretty simple.

To learn more about PHRF and the 2008 hotlaps, go here for detailed summary of the results.
If you want to see the 2008 discussion, there are twelve pages here.

You asked a specific question, however; how to use the handicap numbers. Here's how:

1. Let everybody race the way they always do (grin). They all have finish times.
2. Take the most recent Handicap list. I've posted it below from last year.
3. Multiply each boat's finish time by the 'handicap factor' for that boat class. That gives you a new, 'corrected' finish time.
4. Use the corrected finish time instead of the actual time.


For example: Three boats race, a Tako3.3, a Tahiti Ketch I, and a Tetra 35v1.2. They all cross the finish line together with a time of 03:00. Who won?
Change 3min to 180sec in order to make it simple,
then find that boats 'handicap factor' on the table. Use the last column labelled PHRF with numbers in red. All the other columns just show where the numbers came from.
Now multiply each boat by its correction:

TAKO:
180sec * 1.35 = 243sec

Tahiti Ketch I:
180sec * 0.86 = 155sec

Tetra:
180sec * 1.12 = 198sec

Thast gives you corrected times for the boats. The boat with the fastest time wins; that's the Tahiti Ketch I above. That makes sense; any Tahiti Ketch skipper who can cross the finish line together with a TAKO deserves to win; its a much slower boat.

Unfortunately, many boats were not around for testing last year and are not on the list. That's why we're repeating everything now in Madaket. The madaket line numbers and handicaps I posted this moring are VERY preliminary and almost certainly will change as we collect newe data. So PLEASE use the numbers below if possible. If you absolutely need a number for a new boat, grin... hold your nose and use the table from this moring. By next week the numbers will be much more stable.

[Tig Spijkers]

Thank you again Jane, for another detailed explanation.

What springs to mind and I am sure all RD's will agree with me here is the hectic and excitement during and most of all after a race.

To start the next race, but most of all what is the score!

It would take to much time to run the show and calculate all the results in the same time.
I've tried doing it during the second race, but then you loose the chance of being able to participate or observe the racers.

There would be two solutions in my oppinion.

1. An RD hud that can calculate the results, still alot of manual input from the RD.

2. Race lines that not only recognise the registery number, but also the typ of boat.
(as an optional setting in the menu, ie. PHRF On / Off)

Solution 2 would be perfect in my oppinion.

[Jane Fossett]

Thank you again Jane, for another detailed explanation.

What springs to mind and I am sure all RD's will agree with me here is the hectic and excitement during and most of all after a race.

To start the next race, but most of all what is the score!

It would take to much time to run the show and calculate all the results in the same time.
I've tried doing it during the second race, but then you loose the chance of being able to participate or observe the racers.

There would be two solutions in my oppinion.

1. An RD hud that can calculate the results, still alot of manual input from the RD.

2. Race lines that not only recognise the registery number, but also the typ of boat.
(as an optional setting in the menu, ie. PHRF On / Off)

Solution 2 would be perfect in my oppinion.

Ummm...
grin
is that all you want?
I want a lot more!

Tig:

1: If a skipper has a "#" in the boat object's name field, the line will automatically enter the owner's name (not necessarily the skipper). It will also enter the 'type' of boat (what's included in the boat name info).
2. It's true, the sailor needs to enter the name of the course, the line can't guess that. We're making this easier with a drop-down list, however. You only need to worry about this is you decide to save your lap data.
3. You want the database to do the PHRF math conversions for you? Grin... Yo got it! That feature's been part of the line since last July, when Cynthia tested it out on the Heros sims for Sailing for Life. The problem of course is that new boats do not have good PHRF handicaps yet... but they will very soon.
4. You concerned RD's have lots to do keeping track of the numbers while watching the fleet and starting the next race? Well the database will remember all the race results for several days, even if no information is saved. We haven't yet set up "Race Director" permissions to conveniently review regatta results, but that seems pretty a obvious use. You wont have to write anything down or save it during the race itself, and all the race data would flash up on the screen for the fleet's approval too.

RD features sound line a great idea!

(Was that everything on your list?)

[Cynthia Centaur]

Another problem indeed is, that the line can *not* detect the boat type but ony the object name.

As this is the same information people see in their inventory people tend to modify this information. So you often find lines in the database like: "My nice twenty all customized sails #33".

Sure a human can correct that in many cases but an automated detection of boat types and following an online calculation in real time of PHRF correction factors seems illusionary. This is also the reason why corrected times don't shine up for most races currently. Nobody did the work to correct all these funny boat type lines that occured in the meantime.

Similar problems exist with the "skipper" of the boat as the line is not able to detect persons sitting in a boat with the reliability needed.

Maybe we should define a "boat to line" protocol that includes the necessary data?


best regards
Cyn

[Tig Spijkers]

For now I suppose that was it on my list

Thank you Cynthia for your detailed explanation.
I am sure you guys are working on this and allready have possible solutions.

I would have thought a simple line/ protocol in the discription or boat name would be sufficient.

About the line not being able to detect who is sitting on it, it does detect the owner of the boat.

I am sure the line can detect the owner and even group if needed (for team races, add group detect to my wish list dear Jane).

I simple protocol for how the discription of every boat should include type and registery number should solve the line being able to detect and calculate the PHRF.



Thank you

[Jane Fossett]

Here's a quick spreadsheet and summary table of the current Madaket PHRF Hotlaps results.
Twenty-five skippers sailed 192 laps in 19 different boats.
I'll discuss the results at greater length later, but it looks like many of the numbers are starting to fall into place. For example, the T20 and knockabout have scores that are virtually identical to the 2008 PHRF data. Simply by eyeballing the performance graphs last year I thought the Knockabout was 10-15% faster than the T20, at least on paper. That's confirmed by the numbers below.
Unfortunately, there is still substantial statistical variability in t dataset, and so I don't think many of the other numbers are too reliable. For example I think the tako is clearly faster than 1.25 (it scores around 1.35 -1.38 on all the other tests). The same thing goes for the Larinda, which got a 1.02, the same speed as a defender or yawl. All the prior testing shows the Larinda is about 10% faster than the yawl; I think most sailors would agree with that as well.

OK, we need more lap data!!!

[Jane Fossett]

Here's the latest summary of phrf handicaps. The numbers are sliding into place.