Web Conferencing Blog

Normally Blackboard Collaborate wouldn’t have made it into our ranking. The vendor fails to deliver one element we see as crucial: making pricing information openly and easily accessible. Even searching the Blackboard website for quite some time left us none the wiser in that regard. Now there are two reasons we decided to test and rank Blackboard Collaborate after all:

1. We were curious to see how the successor of Ellumiate vRoom would do, a tool previously ranked 18 in our comparison of online meeting tools.
2. The renowned IT-research organization Gartner Inc. lists Blackboard as one of only 12 other vendors among the illustrious gang of Adobe, Microsoft, IBM and AT&T. So the tool must be good, right?

Wrong! After our extensive tests we were surprised to see a phenomenon that usually occurs the other way around. Web conferencing tools tend to get better with every version upgrade and usually climb up a bit in our overview – if only temporarily. Blackboard Collaborate disproved this trend and crashed all the way to 33. The only tool ranked lower is Microsoft NetMeeting, which we really only keep in our ranking for comparison and for which development and upgrading has stopped years back.

So did the programmers do a bad job after Blackboard acquired Elluminate and tried to integrate the vRoom-platform with the Wimba-technology? Well no, we wouldn’t go that far. For basic ‘learning management’ scenarios in which there are clearly defined roles for the participants Blackboard Collaborate works just fine. You can see that the platform has developed from the very specialized field of Learning Management Systems (LMS). If you are looking for a tool that allows you to be flexible in your diverse collaboration scenarios, Blackboard’s tool fails to deliver in areas that are crucial. Just to give you some examples:

• There is no invitation functionality integrated into Outlook.
• Switching control of keyboard and mouse was not possible in our tests – even though this feature is mentioned on the vendor’s website.
• When setting up a meeting the tool does not create a meeting specific ID which means that participants from former meetings can join any time.

Blackboard Collaborate disproves the phrase of “the name says it all”. The product name “Collaborate” promises a set of functionality and a user experience the tool fails to deliver. Blackboard should consider sticking to its strengths and concentrate on the LMS-market in which we would say it  has a good standing, and rightfully so. And the IT-researchers from Gartner should perhaps reconsider their selection of vendors for their “Magic Quadrant for Web Conferencing” since for Blackboard it seems quite arbitrary.

A couple of weeks ago we mentioned in a post that we would have a look at some tools that are featured in Gartner’s Magic Quadrant but not in our ranking, since those specific web conferencing solutions did not comply with our basic criteria of evaluation.

Gartner and also Forrester apply their own criteria when testing online conferencing solutions. But are they really valid? Take one of Forrester’s vendor select criteria: To be eligible for testing, a vendor must have an annual revenue of over $500 million. Now really, does it matter to a user of an online conferencing tool what kind of revenue the vendor makes? We don’t think so. The most important factor is that the solution performs well. Some small vendors outperform the big players on the market and a comprehensive study of web conferencing solutions has to acknowledge this.

Gartner approaches the web conferencing market in a way similar to Forrester. Gartner assesses the financial health of the provider and the placement of the web conferencing tool in question within the organization’s product portfolio. The marketing strategy of the vendor is also ranked high priority. These criteria lead to the result that AT&T Connect is ranked better than Intercall Unified Meeting. In our tests we have found that the latter easily outperforms AT&T’s online conferencing solution.

Performance is key. If a tool is user-friendly and offers all necessary functionalities, it doesn’t matter what kind of market power is behind it, as long as the vendor is able to provide customers with reliable service. And there is absolutely no reason to assume that a small vendor cannot do this just as well or even better than one of the big players.

You can check out our comprehensive study of the web conferencing market here.

With it’s Magic Quadrant for web conferencing Gartner offers a comprehensive study of the web conferencing market which includes a detailed evaluation of online collaboration tools. Now we have been approached with a justifiable question: Why do their results differ from ours?

The answer lies within another question: What are you looking for in a web conferencing solution? Which brings us to the criteria of evaluation. Gartner takes a very close look not only at the software itself but also includes the vendor in its evaluation. Overall viability of the vendor e.g. has high priority. Here Gartner assesses the financial health of the provider and the placement of the web conferencing tool in question within the organization’s product portfolio. The marketing strategy of the vendor is also ranked high priority.
This prioritization naturally favors the big players who have the high marketing budgets and are financially better situated than a small vendor who just introduced his 1.0 version to the market.

We have a different focus when analyzing and evaluating web conferencing solutions. The web conferencing solutions we evaluate should e.g. offer an approach different or complementary to what Gartner calls the on-premises-model. Users should not be required to install additional hardware or software to their IT-infrastructure to run an online collaboration solution. We give vendors a higher ranking if they enable you to run the software without great preparatory effort (SaaS – Software as a Service). This is important to us since we test with small to medium businesses in mind, who often cannot afford the costly and time consuming installation. You can find a detailed description of our testing approach and evaluation criteria here.

Gartner also features tools that are not included in our ranking. Most of these tools have been rejected by us previously because they did not comply with our basic requirements, e.g. offering a free test version and pricing information. We will definitely take another look at AT&T Connect, Intercall Unified Meeting, and PGi Netspoke to see if they can be included in our ranking now.

If you know of other tools that we have not tested so far and you feel fit all our criteria for evaluation please let us know. We will definitely have a look at the suggestions.