I decided to mess around with Curl to see how much data it could really handle. In a fit of curiosity I implemented some benchmark tests that are so unscientific they would make Descartes roll over in his grave. I ran the tests on my MacBook Pro on Windows XP through VMWare Fusion. Yes you read that right - not exactly the most preformant configuration in the world. Despite my less than stellar runtime enviroment I was really pleased with Curl's performance.
First I loaded up 10,000 records into a table. The table was a CSV file. Each record represents a contact and has five columns (i.e. full name, email, work-phone, home-phone). To get to 10k records I wrote a Java program that generates the records using UUIDs that are converted into strings for values of each field. I trimmed the string values to what I thought were appropriate sizes for various fields (e.g. 20 characters for name, 13 for phone numbers). You can get the Java program here.
I ran an example PIM program that comes with Curl which was already able to read a CSV file and load it into a Curl visual table widget. The Curl application launched and loaded the 10,000 records quickly. It was able to sort on any of the fields - none of which are indexed - instantly. Scrolling through the records was also instant - no delays at all.
Next I generated a CSV file with 100,000 rows of random records. Curl launched and loaded these records in about 4 seconds. It was able to sort on any column in about 2 seconds and provided instant scrolling through all 100,000 records.
Next I generated a CSV file with 1 million records. Curl took about 2 minutes to launch and load and about 25 seconds to sort on any given column. Surprisingly scanning through the 1 million records by moving or clicking on the vertical scroll bar was instantaneous. I have no idea why you can scroll through 1 million records with no latency but sorting them takes 25 seconds.
Finally I met my match when I generated a CSV file with 10 million records. The Curl application never launched thus ending my experiment - I actually fell a sleep waiting for the Curl application to finish loading, which it never did. I'm not sure what part of the system got hung up. It may have been Curl, or the Firefox browser, or the funky MacPro/Windows XP/VMware Fusion configuration I'm using. It's a probably a combination of all of these things.
I would love to see how other people's favorite platforms handle 1 million or even 10 million records. It looks like Web Dynapro is the current leader but I think Curl may have given it a better run for its money on a faster configuration. How many records can your favorite RIA platform handle?
]]>I enjoy doing these kinds of extreme architectural experiments because it does make you think differently as a programmer and I love giving server administrators heart attacks.
So I was again inspired and wanted to give 10million rows a try. It resulted in a crazy big session state (not surprisingly) but did complete.
http://flickr.com/photos/tjung/2743001410/
I have the unfair advantage of a smallish server running my DB and App. server though. I'm pretty impressed that Curl was able to chew through a 1 million row CSV given the hardware you had it running on.
As to your question about Web Dynpro being RIA, Fat, or Fit client - it is closest to the last one. Web Dynpro produces a metadata based description of the UI and then different rendering engines can be applied to the same application at runtime. The video from my blog is the AJAX/DHTML rendering engine - but we also have an Adobe Flex based rendering engine and a .Net Desktop Client based rendering engine.
]]>Standard Flex DataGrid component also quite fast.
I worked with WebDynpro sometime ago. As I remember, it used some kind of ActiveX, so it's pretty much obese :)
Web Dynpro's main UI libraries don't use any ActiveX. The UI elements are implemented in AJAX and DHTML. We do have a couple of specialized controls - for example Microsoft Excel Integration and multiple file uploads - that use either ActiveX or Java Applets. These are pretty specific use cases however. Even the Adobe Interactive Forms elements have been changed from ActiveX to JavaScript based. The future direction for UI elements that can't be accomplished is JavaScript/HTML is to use the Islands technology that allows us to embed Flash/Flex and Silverlight directly into Web Dynpro.
]]>It's really hilarious to me to see plugin vendors make so much noise about performance and then be matched or surpassed by Ajax.
Ajax, Flash, Java and Curl scale to 1M+ rows. Now can we stop talking about naive benchmarks like this, because it would be foolish to load this much data up front in the types of applications these platforms target.
Real-world performance in a WAN environment comes from really intelligent data management, which reduces server round-trips, thereby reducing network traffic and database hits.
If you care about performance, evaluate a platform's data binding architecture and it's facilities for re-using already loaded data, managing caches, and performing client-side operations where possible. If you could benchmark, say, the Cairngorm architecture in Flex against SmartClient's (Ajax-based) databinding architecture, you'd find very large performance differences that actually affect real applications.
]]>Thanks for commenting on the blog. I would like to see a demonstration of SmartClient handling 1 million records. Can you post an example? You say that Ajax surpasses the plug-ins - I would like to see an actual demonstration.
All the best,
Richard
]]>