Web Conferencing Blog

When we last tested Lync Online the tool ranked 15th in our comparison of online meeting solutions. Back then the software had just passed the Beta phase so we wanted to now see how the online collaboration tool had been advanced over time.

Our first finding: Lync Online is not meant to be used as a standalone self-service web conferencing solution and setting it up requires substantial IT expertise. It may sound unproblematic that you need to open and configure your “personal” Office 365 cloud instance to run Lync Online, but we find Office 365 to be a very complex SaaS platform –  at least for those of us who are not IT professionals or well-trained Microsoft partners. So why did Microsoft replace their easy-to-setup Office LiveMeeting solution with a solution which forces end-users to rely on IT professionals, before they can start to use it? Why did they tie Lync Online to Office 365? This makes sense if you consider what we understand to be Microsoft’s goals for Office 365:

• Sell Lync Online along with Sharepoint Online and Exchange Online as hosted solutions to small to medium sized businesses for whom the server versions of Exchange and Sharepoint are too expensive or complex to install and administer.
• Use its dominance on the PC desktop and Windows server to promote unified communications as USP of its web conferencing solution to enterprise customers.

If you look at our test approach you will find that Lync Online doesn’t really fit in with the other tools we have tested and evaluated since we have a different focus. One of our selection criteria for online meeting solutions is that no IT support is necessary, i.e. that solutions can be set up by anyone with a couple of clicks.  But setting aside the setup aspect we still wanted to know how the tool performs and set it up and configured it for our purposes. In doing this we basically had two choices of setup scenarios. We either could host our emails with Exchange Online to be able to use Lync’s Outlook functionality or we could integrate our own email domain via “federation” in Office 365.

Without knowing how to change name server settings to link your organization’s domain into Office 365 you are lost. If you want to integrate an existing domain into Office 365 without having all domain services (esp. email) transferred the online help provides a confusing set of help documents, not organized along the setup workflow at all. But again, this is simply due to the fact that we were trying to setup Lync Online in a way it is probably not intended for. To set up Lync Online without being integrated into a larger corporate network requires an experienced IT administrator and even for them it is no piece of cake. And it takes a while to get everything up and running since altering domain settings is a matter of days.

Eventually we did manage to get everything set up and conducted our usual tests. We tried to get a meeting going between a host who has an Office 365-based Lync Online user account and an external party with nothing preinstalled thus checking the functionality of the Lync web client.  We discovered that the front end isn’t very user friendly. E.g. when the host shares his screen the participant needs to select “show stage” from the share-menu to be able to see it which is not very intuitive. The menus as a whole are not arranged too well and things we consider simple such as changing presenters are more complicated than they need to be.

Lync Online does offer most of the functionalities we expect from a good online collaboration solution, however. You can share your desktop or single apps, upload images and presentations and use the whiteboard functionality. Chatting with other participants is also possible, although if you are not within the same network as all other participants there is no private chat available. When using only the web client video is not supported. External users rather need to download the attendee app, which then gives them full functionality.

So what did we walk away with from our tests? The most basic observation is that Lync Online has its strengths in a unified communications scenario within an organization’s IT infrastructure which aims to seamlessly integrate audio, video and web conferencing, instant messaging and document management. To use it as a standalone solution for web conferences with external participants does not really pay off if you consider the high installation effort and configuration of the solution which requires expert help.

Next to the complexity of setup the sheer time that passes between registering and being ready to go is too long. It takes days to have all the necessary DNS changes in place which is a stark contrast to the few minutes it takes to get going with other tools and the easy one-click setups many of them offer. When testing the Beta version we had outsourced the setup to an IT expert so that we could focus on the functionalities of the solution. We got the feedback that setting up the tool for our purposes was rather complex and decided not to let this influence the ranking of the tool then. Now, with the new version, the setup factors into the overall performance review and as a consequence Lync Online moves down in our ranking, from position 15 to 22.

As a comparison, Microsoft’s Office Live Meeting 2007, which has already phased out is ranked much higher since it is a sleek and easy to use tool that can be utilized outside a company’s IT-infrastructure for convenient online collaboration.

Normally Blackboard Collaborate wouldn’t have made it into our ranking. The vendor fails to deliver one element we see as crucial: making pricing information openly and easily accessible. Even searching the Blackboard website for quite some time left us none the wiser in that regard. Now there are two reasons we decided to test and rank Blackboard Collaborate after all:

1. We were curious to see how the successor of Ellumiate vRoom would do, a tool previously ranked 18 in our comparison of online meeting tools.
2. The renowned IT-research organization Gartner Inc. lists Blackboard as one of only 12 other vendors among the illustrious gang of Adobe, Microsoft, IBM and AT&T. So the tool must be good, right?

Wrong! After our extensive tests we were surprised to see a phenomenon that usually occurs the other way around. Web conferencing tools tend to get better with every version upgrade and usually climb up a bit in our overview – if only temporarily. Blackboard Collaborate disproved this trend and crashed all the way to 33. The only tool ranked lower is Microsoft NetMeeting, which we really only keep in our ranking for comparison and for which development and upgrading has stopped years back.

So did the programmers do a bad job after Blackboard acquired Elluminate and tried to integrate the vRoom-platform with the Wimba-technology? Well no, we wouldn’t go that far. For basic ‘learning management’ scenarios in which there are clearly defined roles for the participants Blackboard Collaborate works just fine. You can see that the platform has developed from the very specialized field of Learning Management Systems (LMS). If you are looking for a tool that allows you to be flexible in your diverse collaboration scenarios, Blackboard’s tool fails to deliver in areas that are crucial. Just to give you some examples:

• There is no invitation functionality integrated into Outlook.
• Switching control of keyboard and mouse was not possible in our tests – even though this feature is mentioned on the vendor’s website.
• When setting up a meeting the tool does not create a meeting specific ID which means that participants from former meetings can join any time.

Blackboard Collaborate disproves the phrase of “the name says it all”. The product name “Collaborate” promises a set of functionality and a user experience the tool fails to deliver. Blackboard should consider sticking to its strengths and concentrate on the LMS-market in which we would say it  has a good standing, and rightfully so. And the IT-researchers from Gartner should perhaps reconsider their selection of vendors for their “Magic Quadrant for Web Conferencing” since for Blackboard it seems quite arbitrary.

Last weekend, the biggest update in the six-year history of webconferencing-test.com went live. Almost all aspects of our portal have been reworked to enhance the value and the usability of the site:

• Compare Tools Pro now allows visitors to compile a tailored dossier of data from our 30-plus tool tests, offering detailed information on over 80 functions and criteria for all solutions. The service (which is subject to a charge) supports organizations in simplifying and accelerating the expensive and time-consuming process of evaluating web conferencing platforms.
• The new Compare Tools Free option offers initial guidance for those searching for the right software. As the name suggests, this service is free of charge. Portal visitors can select the tools they are interested in and compile a compact, comparative overview of test results for their shortlist. Aggregated results are given for the following categories: functionality, usability, meeting setup, software installation, cost transparency, security, and system requirements.
• What’s more, webconferencing-test.com now includes a guide on how to select the right solution (Evaluation Guide), and new test criteria for video- and audioconferencing functionality. A new, featured ranking offers a quick overview of the videoconferencing functionality of each web conferencing solution.

Why have we re-launched the portal?

When webconferencing-test.com first went live back in 2006, we began by testing the five most popular online meeting tools and presenting the results in a table. Since then, the number of virtual meeting offerings we test has risen to over 30. Our goal was to bring transparency to this market – which means ensuring we consider as many solutions as possible.

As the years went on, our portal was limited by one key factor: Only results for the top five tools were shown in detail, broken down by specific criteria. We just assumed that for the majority of users, complete results for the five best-ranked platforms would be enough. Why should we deal in detail with those that didn’t make it into the top five?

A flood of feedback from portal visitors told us why. As it turns out, there is significant interest in accessing in-depth test results for lower-placed tools, too – for a number of good reasons. Perhaps you have already worked with a certain tool (and would like to see how your experience compares to our findings), or maybe you are looking for very specific functions, which are provided by one of the tools ranked 6th or below (such as web conferencing software for visually impaired users, for example).

Moreover, it has become clear to us over time that it is not just self-employed professionals and smaller companies that use our website as a starting point for selecting a web conferencing platform – many larger organizations do, too. The feedback from these enterprise users can be summarized as follows: our portal saves them the time and effort of identifying all the relevant players and painstakingly compiling the facts and figures for each provider.

But for these larger organizations, too, the fact that detailed results were only given for the top five solutions limited the usefulness of our comparison. In fact, when it comes to enterprise implementations, there are special criteria – that we could not take into account – that often play a key role. For example, companies may wish to shortlist a provider who is already a preferred supplier, or a solution that can be easily integrated into the existing IT infrastructure. Prime examples include IBM (in 11th place at present) and Microsoft (ranked 8th and 17th).

If our analysis of the benefits and limitations of the previous incarnation of webconferencing-test.com stands up, then the introduction of the Compare Tools Pro service should significantly boost the attractiveness of our portal.

But why don’t we offer this enhanced service free of charge? After all, commercial considerations were not our main motivation when we originally launched our portal. There is a simple answer. This is primarily an experiment to find out how willing professional web users are to pay for valuable content on the Internet. Moreover, if the results are positive, we will naturally be glad to recoup some of the considerable ongoing investment we make into conducting and extending our tests.

We hope and believe that we are offering our visitors excellent value for money. Our approach is the opposite of the conventional model for generating revenues with advisory content. IT research and advisory companies such as Gartner exploit their high brand awareness and reputation to sell their research results at a steep price.

Take the following example: Gartner sell its annual “Magic Quadrant for Web Conferencing” report (18 pages) for US $1,995.00. This publication deals with twelve web conferencing providers, without a detailed analysis of the functionality and usability of the solutions. As a niche portal, webconferencing-test.com may not enjoy the same brand awareness, but offers far more detailed content on the facts and features of each web conferencing solution. If you want to buy the test results for all 34 tools we have published information on, it costs just US $69.50 during the introductory period, and US $139 after that. And in reality, the vast majority of visitors pay much less. For example, if you already have a shortlist of four “favorites” and want a quick, fact-based comparison, you can generate this dynamically for just US $9.50.

So you only pay for what you actually need – sounds like a fair deal, doesn’t it? We look forward to seeing whether the 10,000 visitors to our website every month agree. And we would be delighted to hear your feedback on our brand new portal.

A week ago we became a registered Cisco partner – after unsuccessfully trying back in 2009 – and were quite happy about this. We needed some help for details regarding the creation of WebEx trial account links and were invited to a web conference with Cisco partner support – naturally via WebEx Meeting Center. We had an online meeting from our office in Frankfurt, Germany with a US-based representative and after further evaluation with a Cisco European Partner Advisor professional based in Portugal.

In both cases we were amazed to see that during screen sharing there was a latency of up to 5 seconds at some points! This naturally made communication a bit complicated since we needed to jump between lots of web pages on Cisco’s Partner Central extranet and never knew what the Cisco service desk was currently seeing.

As we have said before such latency can have various causes (and often it will be a combination of factors, e.g. slow local internet connection, insufficient compression of screen content by the software, limited backbone connectivity or server capacity on the part of the web conference provider, or the physical latency caused simply by large geographical distances). Since there are so many variables in this scenario, part of them not controllable by the web conference service provider, we have so far not included this as a criterion in our tests.

We feel that this needs to change: Considering that latencies of more than 1 second can impact an online meeting much more than ‘soft’ interface usability criteria or the ‘completeness’ of features and functions, we will develop a testing infrastructure that allows us to collect comparative data to make reliable statements about whether an online collaboration tool is comparatively slow or not.

After our meeting experience with Cisco we cannot help but be amazed: Of all vendors Cisco – as world leader in ‘all things connectivity’ – should have a powerful backbone connectivity that enables a seamless meeting experience. Since our internet connection was at full speed the day we had the two online meetings the question remains how well Cisco WebEx is suited for our usage scenario, which is using the Mac-client and sharing a screen with 1920x1200px. The new version of WebEx Meeting Center is about to be released – including HD video conferencing – and we will see if latency remains an issue.

In June we included a brief survey on webconferencing-test.com asking for the reasons visitors come to our site, how they got there and if they got what they expected. We have gotten quite a number of responses – thank you all! – and would like to present what our visitors had to say.

Looking at the reasons visitors came to our site revealed the following:

Most visitors (31%) were on the hunt for generic information about web conferencing to get familiar with the technology and different ways of online collaboration. 18% were on the verge of purchasing a solution and checked our site to confirm their buying decision. And we were happy to see that 8% of the visitors were current users of a web conferencing solution seeking tips and tricks on using the software in question. In addition, 21% chose our site to begin their search for a web conferencing solution and 9% were looking for news on the online collaboration market. The remaining 13% had their own reason to visit our site…

Of course we got many suggestions for tools to include in our tests, such as ICU Live! from ICU Global. We always appreciate such hints since the market has expanded greatly and it is impossible to keep track of all online collaboration software. The feedback we got relating to this issue, i.e. that our overview is not complete is correct. But some tools don’t appear on our list for the following reasons:

• Target group: If an online meeting tool vendor requires you to make changes to your IT infrastructure the tool is very likely not for our focus group – small to medium sized businesses or the self-employed – who are best of with ready-to-use solutions.
• Popularity: Finding a tool that works for you is great. But if you are one of the 10 total users it just may not have an appeal for a broader audience and we need to prioritize our resources when it comes to testing.
• Performance: Some tools are simply bad. Full stop. No need to include them in our ranking.

Another bit of feedback was that our pricing info was not detailed enough and that we should incorporate pricing models for more usage scenarios on our site. Availability of pricing information is an important factor for us as we also state in our test approach. That being said there is no way for us to depict all the different usage scenarios that might come into play for the various customers. What we depict is the overall cost model we find on the vendors’ websites and if you have special requirements you will be better of contacting the vendor of your choice directly.

Thanks again to all participants for the helpful feedback. We will continue to try and state all the information on online collaboration tools you might need and appreciate your feedback should we have missed anything.

Many companies currently still use NetMeeting. However, the tool’s shortcomings make it very inconvenient for continued usage which is why those companies are now on the lookout for an alternative web conferencing solution.

Incompatibility with Microsoft Office 2010 is only the last of issues that have made usage of NetMeeting quite tedious. If an online collaboration tool does not allow you to demonstrate a simple PowerPoint presentation there is something seriously wrong.

The whole issue started a while back when Microsoft decided to no longer support NetMeeting with the release of Windows Vista – NetMeeting had been a part of the Windows OS since Windows 95. After Vista came out companies using NetMeeting found workarounds to keep the tool running but now even those solutions will no longer work.

Why did NetMeeting become so popular in the first place one might ask. Well… it was there. Simple as that. Since it was integrated in Microsoft’s OS companies made use of it for their internal online collaboration even though the tool never really was good. So the end of life offers a twofold opportunity for current NetMeeting users:

• Companies can select an online meeting solution that enables collaboration to go beyond the firewall and allows for meetings with external participants without compromising security.
• Prospective tools can be evaluated for the exact functionalities needed to guarantee ease-of-use.

Online collaboration tools come in many colors and it is essential to look at the basics before purchasing a specific solution. Would a unified communication solution such as Lync be best or rather a standalone solution such as LiveMeeting? And then of course there is the question of hosted vs. on-premise.

With its demise NetMeeting poses some serious challenges to the companies that use it. On our site you can read about the different approaches the vendors of web conferencing solutions take.

You know this feeling: You are sitting somewhere and your eyes wander back to one and the same person and all of a sudden you realize ‘hey, he looks exactly like …’. Well, we had that exact same feeling when we tested ClickMeeting. We were checking out the new video functionality and all of a sudden we realized ‘hey, this looks exactly like Adobe Connect’!

Well it sure does. ClickMeeting is pretty much an exact copy of Adobe Connect 8. And ClickMeeting is a good copy, too. One of our testers found it a bit complex but really the functionalities offered can be arranged on the screen nicely to make sure participants are not left confused by the various little screens you can use. The new video functionality worked really well, too. The single webcams were resizable and video quality was also good.

But there is a reason why ClickMeeting ranks lower than Adobe Connect 8. Unlike with Adobe Connect 8 there is no Outlook integration for ClickMeeting hosts. The missing of marking tools in screen sharing mode also makes you aware that ClickMeeting currently doesn’t offer what online meeting participants should be able to take for granted. Therefore we cannot rank the tool higher than number 9.

Cisco, RHUB and Microsoft – the web conferencing vendors ranked numbers three through five – will have to each step down a rank. BeamYourScreen, formerly ranked number seven, is now available for download in version 4 and brings many enhancements. Just as with the release of version 3 last year, when the tool climbed from rank 18 to 6, the developers once again have done a great job in actually improving the software.

Version 4 of BeamYourScreen comes in an all new layout which is nice and lean and makes for intuitive usage. Users can also adapt the range of functionalities for every single meeting. What we weren’t too exited about is the minimum 12 month contract duration. There are other small disadvantages, too but overall the web conferencing solution has been greatly improved and thus only stops short of knocking Citrix GoToMeeting and Adobe Connect 8 from numbers one and two.

If you decide to try out the new version of BeamYourScreen let us know what you think of it and post your experiences.

Citrix, Adobe and Cisco: Three of the biggest online meeting solutions vendors have now integrated video functionality into their solutions. We checked them out and here is what we found:

All three tools offer not only video but rather HD video. And yes, the quality is very impressive. Citrix GoToMeeting limits the number of webcam participants to 6 unlike Cisco WebEx which allows up to 500 participants or Adobe Connect 8 where you have no restrictions at all. But as always bigger isn’t necessarily better. One downside of a video conference with let us say 20 participants is that you will need a lot of bandwidth to really get that HD quality. And with every new participant that joins the webcams on display just get smaller and smaller. Now we haven’t tried it with 20 participants ourselves but it probably is hard to see who is actually speaking.

Next to the number of possible participants the ease of integrating video into a conference was another important item on our list. All three tools did well when it comes to basic integration and Adobe and Citrix lead the way when it comes to flexibility. Users can resize the webcams on display or place them wherever it is most convenient on their screens.

A nice touch was linking the videos to VoIP, which GoToMeeting and WebEx offer. What does that mean? Well, the webcam of the current speaker is highlighted (Citrix) or centered (WebEx). The centering can give you a headache, though. Just imagine a discussion between 10 participants with frequently changing speakers…

Overall, Citrix really did the best job of integrating HD video. The new functionality is lean and easy to use. It blends into the overall tool very well and covers the scenarios ‘video-only’ and ‘video combined with screen sharing’ best.

But Adobe and Cisco don’t trail by far so if you are used to meeting online with one of the two you are very likely to be satisfied with the video conferencing option either one offers.

BeamYourScreen version 4 went live today as planned. However, we haven’t updated our ranking yet. We have tested the tool and formed our verdict but when we double checked the also new website today we noticed that there currently is no pricing info on display. We contacted BeamYourScreen and were told that the information will be included on the site shortly.

Since we regard openly available pricing information on vendors’ websites as a crucial point in our evaluations we will wait with updating our ranking until it is fully displayed on the BeamYourScreen site.