512000 concurrent websockets with Groovy++ and Gretty

We are staying in front of new world - all major browsers either support already or plan to support in next major version HTML5 (not in scope of this article) & WebSockets (main subject of the article). In 6 to 9 months we as application developers will have in our hands extremely powerful client side tools to build new generation of the Web. But are we ready on server side? And if not, what the point in having powerful and reliable communication channel between browser and server and non-utilising it.

In this article I will talk about my expirience of handling 512K concurrent websockets using Groovy++ & Gretty.

Why 512K and not 1M?

This is very fair question and my initial goal was to handle exactly 1M concurrent websockets on one machine. It seems that due my lack of knowledge of tuning TCP/IP on Linux I was not able to achieve more. I had enough free CPU power and a lot of free memory but after 524285 open connections (always the same number) the server stopped to accept new connections.

The magical number 524285 is so close to 524288=512*1024 that I can guess (and only guess) that we deal either with some limitation of Linux setup or with something in settings of Amazon network infrastructure (where I run my tests) .

So what was the experiment about? In general it was my way to estimate (or at least to have feeling) how many (virtual) hardware units do we need if we hope to handle A LOT of concurrent users.

The server itself is very simple. It does the following:

• On HTTP request from a client it responds with text document containing Groovy script to be executed by client.
• It accepts and keeps websocket connections from clients and respond to every received message (just plain string) with the same string in upper case

The client program (running on separate machine) request the server for a scenario and then compile and execute it.

Why do we need this trick with sending script to client?

Truly speaking we don't. When I started writing the application I had illusion that it will simplify deployment to many clients. In fact, it did not and I had rsync all clients with my development machine after recompilation anyway.

The scenario itself is also very straight forward. Client program opens 64000 concurrent web socket connections and approximately every 25 seconds sends to the server short string (approximately 30 characters). So server need to handle around 20000 requests per second or around 600K/s traffic in and the same amount out.

To emulate 512000 concurrent clients I used 8 machines and 1 machine was the server. All machine was of the same "m1.large" Amazon EC2 instances with 7.5GB memory and 2 virtual cores running Fedora 11. 2.5GB memory was used by Java heap (around 5K per connection but of course not including kernel structures) and total CPU utilization was under 30%

The server is written on Groovy++ using Gretty, which is lightweight server based on brilliant Netty framework and developed as part of Groovy++ standard library. More information on both can be found at Groovy++ home

Gretty is extremely lightweight and fully non-blocking event driven web server. Gretty itself is written on Groovy++ and fully utilize concurrency libabry. It is not servlet container or any other relative of JavaEE.

Right now it supports only

• static files
• http requests (including modern /param1/param2 REST-like requests)
• web sockets (including long-polling emulation protocol for old browsers)

Here is essentially the whole server code. Obviously it is statically typed Groovy++ code.

        GrettyServer server = [
            webContexts: [
                "/" : [
                    public: {
                        get("/scenario") {
                            response.text = """
.............. here is client scenario code ...................
"""
                        }

                        websocket("/ws",[
                            onMessage: { msg ->
                                socket.send(msg.toUpperCase())
                            },

                            onConnect: {
                                socket.send("Welcome!")
                            },

                            onDisconnect: {
                            }
                        ])
                    }
                ]
            ]        
        ]
        server.start()

I don't want to explain more than written in the code above because I really hope the code is self explaiining.

I hope it was interesting. Get Gretty and Groovy++ a try and let us know what do you think.

Till next time.