At the beginning of this week, we updated our web conferencing software rankings to change the way we calculate the German grades for ranked tools and their criteria. The German grades also have the important role of governing the amount of stars (or "dots" for criteria) a tool receives, and so this change directly influences the visualization of our rankings for all users.
Why did we do this? Well, as it so often happens when a project, such as our website, grows larger and more professional over the years, parts of its core that served fine in the beginning reach their limits and grow obsolete. This is what we felt was the case with our old conversion formula, and it was not without its faults. Here is a graphical comparison of the old (blue) and new (red) conversion functions.
As an example of the old function's shortcomings, we felt that a solution having around 92-95% of our tested features and criteria is an excellent web conferencing tool and should receive the full 5 stars but this was not the case. A rating of around 96% was required, which we felt was a little harsh, and resulted in none of our tested tools receiving 5 stars.
At the other end of the scale, due to the nature of our tests, pretty much every professional tool in existence manages a rating of at least 65%, but this does not mean it’s necessarily a good solution, relatively speaking, but merely that the basic functionality is in place. As a result, we felt that such a rating should result in no more than 1 star, and anything below 60% should receive no star at all. Our old formula gave 2 stars for ratings around and above 60% and 1 star for ratings as low as 45%, which we feel was being too gracious. Another obvious problem was that ratings below 35% resulted in negative values (which got interpreted as 0 stars or 6.0 German grade), and although thankfully none of our tested tools' ratings came close to this point it was just not mathematically "elegant".
A critical area of importance, we feel, is the 70-90% rating interval, where around two-thirds of our tested solutions lie, and we felt that to best help our users, this should be where the stars get awarded; whereas our old formula gave an almost-linear conversion between 35% and 100%, the new formula is designed to focus on the 70-90% interval, differentiating better between the solutions in there.
The rankings themselves have not changed; all the solutions remain in their places with the exact same percentage rating, but the way we visualize this rating for our users with stars or points has changed for the better, allowing an easier differentiation between tools.
For those interested in the mathematics behind the change, the conversion formula and some explanations follow. f(x) corresponds to the number of stars received for a given value x (percentage rating, between 0 and 1.0 (0% to 100%)).
The formula uses the arctangent function to acquire the shape we needed to fulfill all our goals, which is then tweaked by putting the argument x to the third power, in order to move the steepest part of the slope towards the x ∈ [0.7,0.9] region for the reasons mentioned above. The argument is then multiplied by 2π and offset by π to utilize the full period of the arctangent and move its minimum to x = 0. The function is then scaled and offset vertically so, that its minimum (at x = 0) lies at f(x) = 0 and its maximum (at x = 1.0) lies at f(x) = 5 (which corresponds to 5 stars).
Finally, the function is weighted with a factor S (strength-factor) and a linear function f(x) = 5x is added to it with a corresponding weighting (S-1); this is simply a tool to control the function better and to make small tweaks. A strength-factor equaling 1 would result in the linear term dropping out and the curve to be at full strength while a S of 0 will effectively kill the curve and result in a straight, linear conversion. After some testing, we went with a strength factor of S = 0.9.
For practical applications on our website, the result of the conversion is rounded to the nearest integer which then is the number of stars granted (or "dots" in criteria). Our German grades result directly from f(x) as g(x) = 6 - f(x), meaning that a perfect rating of 5.0 stars (or 100%) will give a German grade of 1.0 (the best possible in the German system) while 0 stars would result in a German Grade 6.0 (worst possible).