Home  >  Standards  >  blogs

Forrester "Research" on AJAX Performance?

| | Comments (4)
AddThis Social Bookmark Button
 
forrester_logo.jpg
First of all, let me start off by saying that this is NOT another blog article in the great RIA platform debate. While I am an AJAX developer, and have been for several years now, the point of this article is not to take sides and convince you that AJAX is the best RIA tool available. Because AJAX isn't. Except when it is. You see, tools have uses, and to say that any one platform is in all ways superior is, I think, overstating things. A lot. Yes, I'm a bit biased, but not so much so that I feel the need to convince people that AJAX has no flaws. Flex and Silverlight have strengths that lay against weaknesses in AJAX, and vice versa. The question isn't “Which platform is best”, but “Which platform best fits my needs”.

I say this because when I say that I take issue with Heather Havenstein's “AJAX-powered Web apps disappoint power users, Forrester says” article, I don't want the rest of this article to be miscast. I don't object to this article because it recommends using AIR, Flex, or Silverlight over AJAX. There are valid reasons and circumstances for coming to just such a conclusion. What I have a problem with is using over-generalizations, unsubstantiated statements, and outright inaccuracies in order to push an opinion. These tactics may make for web traffic, but they do not add substance to the matter at hand.

Havenstein's article is not one that evaluates the strengths and weaknesses of a particular platform. Nor is it an article that gives the reader any real insight into the referenced Forrester study, as it offers only a few select quotes and no substantial bodies of text from the Forrester study. At heart, this article does nothing more than use a few selected quotes from the study to support an article. Aside from the vague example of an unnamed “European retailer” who experienced problems with a newly-created AJAX application, and three overly-general categories of applications, no details are given on the applications used in the study. The bulk of this article is the unsubstantiated opinion of a Forrester study, and assumably Havenstein.

The point of the study is that users are disappointed with the performance of AJAX applications, and extrapolated from there that the fault lies with the AJAX platform. These conclusions are, at best, misguided. Misguided because of the fundamental mistake of evaluating a platform based on a sampling of how that platform is used. Even if the underlying platform were the only contributing factor to performance, which is hardly the case, one must remember that an application is the implementation of a toolset for a specific purpose. While the toolset itself does have an impact on performance, the purpose to which that tool is put and the manner in which the application is created have a far greater impact on how the application functions. On Computerworld's posting of this article, commenter Mikael Bergkvist sums this up very well:

“The research doesn't take into account the fact they are judging the apps and not the technology behind them“... “It's a very superficial 'research' - the examples given are more like hearsay than any real research, and I think this is damagingly lazy “

Well said, but allow me to extend this a bit. If a Flex developer creates a large Flex application and declines, for whatever reason, to use the Framework Caching newly available in Flex 3, the result can be an application that takes several seconds to load. However, is this an indication of a problem in the platform or in development practices? The same flawed reasoning can be applied to any platform, because all platforms can be used incorrectly.

Much is made of the performance of AJAX, and the conclusions are drawn that AJAX performs very poorly and that “power users” are disappointed. The major reason given for the supposed poor performance of AJAX appear to stem from an unnamed European retailer's problems with a newly-created AJAX conversion of an existing Visual Basic application. Users were dismayed when complex screens took many seconds, as opposed to the “nearly instantaneous display of the old client app”. Both article and Forrester study appear to lay this at the feet of the AJAX platform, and by implication claiming that AJAX applications perform poorly in general. However Google has a history of creating AJAX-powered applications that perform well. Were Gmail and GoogleMaps part of the study? While how well an application performs can be a very subjective measurement, users certainly do not appear to be so disappointed in these applications that they are clamoring for alternatives. There are, of course, many other examples of highly used AJAX applications, but it seems odd to me to discuss the advantages and disadvantages of AJAX as a platform without so much as referencing the unarguably successful applications of perhaps the highest-profile source of AJAX applications. It also seems odd to me to make the overly-broad statement “The reaction time and general speed of user interaction is always limited with AJAX business applications, the report said “ in light of such well-known examples to the contrary.

In conjunction with the unnamed European retailer, Havenstein indulges in a little Reductio Ad Absurdum and notes that “user interactions may require a round trip communication between the browser and server for each input field. Some large applications could easily have 50 fields on a single screen“ as an illustration of AJAX performance problems. To reference Computerworld commenter “nsm”:

“Apparently someone has VERY BAD UI designers. There shoudn't be 50 fields even in a desktop application. If you have 5 fields that is hardly AJAX's fault. “

Again, well said.

Aside from the flawed correlation between the capabilities of a technology platform and the use of that platform in applications, the article also contains several statements that are, at best, debatable and misleading.

“Forrester said that as AJAX framework vendors work hard to overcome these barriers, they have encountered additional problems. For example, improvements in bandwidth have not led to expected AJAX performance gains. "Bandwidth has widely improved and people are starting to realize that many AJAX applications have not speeded up accordingly," Forrester noted. “

How Forrester came to this conclusion is oddly missing, but for an opposing view- and one that does list its methodologies- check out http://www.webperformanceinc.com/library/reports/AjaxBandwidth/. The truth of the matter lies greatly in how AJAX is used. If you use AJAX with an eye toward saving bandwidth by requesting only what data is necessary from your webserver, rather than refreshing a complex page in order to update a smaller part, your bandwidth savings can be tremendous. That does not mean that the act of using an AJAX library automatically saves bandwidth, however. Again, this article gives few technical details on the applications involved in the study, so any analysis is impossible.

“The problems likely could be overcome if AJAX vendors and browser companies moved in the same direction to fix them, but Forrester noted that the opposite is actually true. Microsoft is investing in AJAX alternatives like Silverlight while Mozilla lacks the "hundreds" of developers it would need to fix the problems. “

The obvious implication here is that because Microsoft is investing in Silverlight, they are not planning on investing in the AJAX platform, and that Mozilla is so strapped for developers that there is no way they can implement such improvements. As accountability is an issue here, let me just say that this is patently untrue. Not only is Microsoft building more AJAX development support into Visual Studio, but all major browser vendors are dramatically improving Javascript rendering performance. Internet Explorer 8, Firefox 3, Safari 3.1, and Opera 9.5 all show vast improvement in rendering Javascript, largely due to Javascript's prominence in RIAs. None of the major browser vendors want the stigma of not being able to properly render the new Web 2.0, and the products of their work is starting to show, to the benefit of web users everywhere, regardless of vendor preference.

Secondly, while I don't know how many developers have contributed to Firefox 3, the above quote, to me, displays a shocking misunderstanding of the term “Open Source”. While such a misunderstanding is perhaps understandable from a market research company, a purported tech blogger ought to understand the implications of an open source project. I don't know, or claim to know, how many developers it takes to create a major browser version, the progress and success of Firefox 3 appears to show that Mozilla has enough developers to remain competitive.

"You might have to implement some business logic twice - in the client AJAX framework and in server-side transactions,"

This is an interesting claim. Earlier in the article, Havenstein uses the claim that

“In addition, because most AJAX frameworks tend to keep all real business logic on the server as opposed to local systems, user interactions may require a roundtrip communication between the browser and server for each input field“

as a reason why AJAX is an inferior platform. Yet later in the article, we're told that business logic may have to be implemented twice. If AJAX is capable of interacting with server-side business logic, and it is, why the later claim of implementing logic twice?

Competition can be good for innovation, and that's as true in the RIA world as it is anywhere else. Competition leads to discussion and discussion leads to debate. And yes, sometimes debate leads to argument. This is all healthy and, when done well, will only strengthen the RIA world. The problem occurs when- for whatever reason- the debate turns from a comparison of facts to mere opinion manipulation. It doesn't really matter whether this article was the result of bias, or merely a writer having an off day. When the desire to push a point overshadows the willingness to properly support that point of view, everyone loses.

Comments

4 Comments

Rob McKeown said:

What I find interesting is that the title of the article specifically mentions "power users". In most applications power users are a small percentage of the user base. Making technical decisions based on a very small segment of your users can be a bad thing. Similarly, not many users of applications are beginners, at least not for very long. So you really need to make good decisions for the intermediate users. I posted more about this here.

Everytime I read about applications with 50 fields on a page, I think about printing out http://stuffthathappens.com/blog/2008/03/05/simplicity/ and sticking it to my wall...

Laurie Gray said:

Thanks so much for bringing this to the the light of day. Forrester's journalistic style has backslid in recent years, and this is further proof. Unfortunately, so many decision makers read Forrester articles that this sort of misinformation will make us do double-time to repair the impressions it creates.

Matt Osbun said:

@Laurie Gray

That's why I wanted to write this. Studies like this do no one any good. Not the developers who sincerely believe in the platforms that they use, and not the decision makers who think that they are making an informed decision.

Not that I believe that there are thousands of IT managers out there making decisions based solely on Forrester studies, but no one voicing a dissenting opinion has about the same effect as when there is no dissenting opinion.

Leave a comment


Type the characters you see in the picture above.

Tag Cloud

Related Books

Development Series

Get an overview of the tools and technologies that work together to allow developers to build Rich Internet Applications (RIAs) quickly and easily.

Anatomy of an Enterprise Flex RIA

Archives


 
 


Or, visit our complete archive.  

About This Site

Welcome to the premiere community site for all things RIA sponsored by O'Reilly Media and Adobe Systems Incorporated.

About Us
Meet the Experts
Meet Our Contributors
Send Us Feedback