Entrepreneur. Creator of Groovy++ Alex is a DZone Zone Leader and has posted 31 posts at DZone. You can read more from them at their website. View Full User Profile

512000 concurrent websockets with Groovy++ and Gretty

07.29.2010
| 44963 views |
  • submit to reddit

We are staying in front of new world - all major browsers either support already or plan to support in next major version HTML5 (not in scope of this article) & WebSockets (main subject of the article). In 6 to 9 months we as application developers will have in our hands extremely powerful client side tools to build new generation of the Web. But are we ready on server side? And if not, what the point in having powerful and reliable communication channel between browser and server and non-utilising it.

In this article I will talk about my expirience of handling 512K concurrent websockets using Groovy++ & Gretty.

Why 512K and not 1M?

This is very fair question and my initial goal was to handle exactly 1M concurrent websockets on one machine. It seems that due my lack of knowledge of tuning TCP/IP on Linux I was not able to achieve more. I had enough free CPU power and a lot of free memory but after 524285 open connections (always the same number) the server stopped to accept new connections.

The magical number 524285 is so close to 524288=512*1024 that I can guess (and only guess) that we deal either with some limitation of Linux setup or with something in settings of Amazon network infrastructure (where I run my tests) .

So what was the experiment about? In general it was my way to estimate (or at least to have feeling) how many (virtual) hardware units do we need if we hope to handle A LOT of concurrent users.

The server itself is very simple. It does the following:

  • On HTTP request from a client it responds with text document containing Groovy script to be executed by client.
  • It accepts and keeps websocket connections from clients and respond to every received message (just plain string) with the same string in upper case

The client program (running on separate machine) request the server for a scenario and then compile and execute it.

Why do we need this trick with sending script to client?

Truly speaking we don't. When I started writing the application I had illusion that it will simplify deployment to many clients. In fact, it did not and I had rsync all clients with my development machine after recompilation anyway.

The scenario itself is also very straight forward. Client program opens 64000 concurrent web socket connections and approximately every 25 seconds sends to the server short string (approximately 30 characters). So server need to handle around 20000 requests per second or around 600K/s traffic in and the same amount out.

Someone can argue if such structure of traffic is realistic or not. I don't want to go in to deep dispute here as it simulates very well one of applications under development in my company.

To emulate 512000 concurrent clients I used 8 machines and 1 machine was the server. All machine was of the same "m1.large" Amazon EC2 instances with 7.5GB memory and 2 virtual cores running Fedora 11. 2.5GB memory was used by Java heap (around 5K per connection but of course not including kernel structures) and total CPU utilization was under 30%

The server is written on Groovy++ using Gretty, which is lightweight server based on brilliant Netty framework and developed as part of Groovy++ standard library. More information on both can be found at Groovy++ home

Gretty is extremely lightweight and fully non-blocking event driven web server. Gretty itself is written on Groovy++ and fully utilize concurrency libabry. It is not servlet container or any other relative of JavaEE.

Right now it supports only

  • static files
  • http requests (including modern /param1/param2 REST-like requests)
  • web sockets (including long-polling emulation protocol for old browsers)

Here is essentially the whole server code. Obviously it is statically typed Groovy++ code.

        GrettyServer server = [
webContexts: [
"/" : [
public: {
get("/scenario") {
response.text = """
.............. here is client scenario code ...................
"""
}

websocket("/ws",[
onMessage: { msg ->
socket.send(msg.toUpperCase())
},

onConnect: {
socket.send("Welcome!")
},

onDisconnect: {
}
])
}
]
]
]
server.start()

I don't want to explain more than written in the code above because I really hope the code is self explaiining.

I hope it was interesting. Get Gretty and Groovy++ a try and let us know what do you think.

Till next time.

 

Published at DZone with permission of its author, Alex Tkachman.

Comments

Deepak Jacob replied on Thu, 2010/07/29 - 9:07am

!

Nick Kijak replied on Thu, 2010/07/29 - 9:31am

Can you provide any links to Gretty documentation? Google is coming up short.

Alex Tkachman replied on Thu, 2010/07/29 - 10:15am in response to: Nick Kijak

It doesn't sound good but Gretty is so new that we have no documentation at all :(

Nick Wiedenbrueck replied on Thu, 2010/07/29 - 10:31am

Sounds awesome!

Mike Heath replied on Thu, 2010/07/29 - 11:16am

Gretty is based on Netty, http://jboss.org/netty, which is very well documented. Netty is an incredible framework. It's really cool to see more people using it.

Radoslaw Berbatov replied on Fri, 2010/07/30 - 2:55am

How did you handle > 500k connections on one machine? Does it have more than one IP? If so does it need other setup, configuration change or tuning (on the application/system/kernel level)? It would be nice if you describe this part of that experiment :).

Lars Tore Hasle replied on Tue, 2010/08/03 - 2:28am

I am impressed! I started the server but was not able to invoke it (http://0.0.0.0/0.0.0.0:8080) with firefox 4. What url should I use?

Alex Tkachman replied on Tue, 2010/08/03 - 10:42am in response to: Lars Tore Hasle

Could you please address this question to Google Group http://groups.google.com/group/groovyplusplus It will be much more natural to discuss it there

Carl Byström replied on Mon, 2010/08/30 - 4:36am

Good work! Interesting to see how far you can push Netty. 512k is quite a good figure but I guess it would vary depending on a lot of factors. We use Netty for websockets too at http://beaconpush.com and it works really well. We even do a mix of traditional of long-poll and websockets. For us, it's quite expensive to setup and tear down the TCP connections coming in. Since end-users often navigate around a site this results in many (re)connections to our servers. This is actually our main reason for when to scale out to yet another machine. Other than that, we've had zero issues with Netty and we also host it on EC2, where you did the tests. Great combo.

Nikhilesh Gargi replied on Thu, 2011/02/03 - 7:18am

1. Were the connection SSL or normal? (2.5GB memory was used by Java heap (around 5K per connection but of course not including kernel structures) and total CPU utilization was under 30%" )

2. What is the connection acceptance rate for the server? i.e. number of connections/second accepted by the server.

Peter Buchmann replied on Sat, 2011/11/26 - 6:46am

Thank your for this great article. I ran into the following problem: If I try to run the last example with the @Typed annotation on the package, I get the following error message:

Can't have an abstract method in a non-abstract class. The class 'example.WebSocketSample$main$2$3' must be declared abstract or the method 'void onMessage(java.lang.Object)' must be implemented.

I don't know how to make the onMessage closure compatible with the abstract methode that is is supposed to override.

If I remove the @Typed annotation, obviously the whole project doesn't compile because of the map -> properties assignment. I am completely new to Groovy++. I am missing out on something?

Thanks for your support and keep up the great work.

Regards from a sunny Zurich, Switzerland

Peter

Carla Brian replied on Sat, 2012/05/19 - 9:35am

This is really a good news.  I will study more on this. This seems interesting. - Instant Tax Solutions Ratings

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.