HTML5 WebSocket defines a bi-directional, full-duplex communication channel operates through a single TCP connection, this article discusses its fantastic performance, the WebSocket protocol principle and its handshake mechanism, and develop a WebSocket application in action (Team Poker).
Embeded youku video link because Youtube is outside of the largest "intranet" in the entire universe!!
HTML5 WebSocket defines a bi-directional, full-duplex communication channel that operates through a single TCP socket over the Web, it provides efficient, low-latency and low cost connection between web client and server, based on WebSocket, developers can build scalable, real-time web applications in the future. Section below is the official definition of WebSocket copied from IETF WebSocket protocol page:
The WebSocket protocol enables two-way communication between a user agent running untrusted code running in a controlled environment to a remote host that has opted-in to communications from that code. The security model used for this is the Origin-based security model commonly used by Web browsers. The protocol consists of an initial handshake followed by basic message framing, layered over TCP. The goal of this technology is to provide a mechanism for browser-based applications that need two-way communication with servers that does not rely on opening multiple HTTP connections (e.g. using XMLHttpRequest or <iframe>s and long polling).
This article is trying to go through WebSocket basic concept, the problems it is going to solve, explain it in essence, watch some experimental Demos, develop a simple WebSocket application in action (Team Poker), and describe current open issues of WebSocket. I sincerely hope it will be systematically, easy to understand, from surface to deep so that eventually readers would not only learn what WebSocket is from high level but also understand it in depth! Any thoughts, suggestions or criticism you may have after reading this article will help me to improve in the future, i would appreciate it if you could leave a comment.
Background
In traditional web applications, in order to achieve some real-time interaction with server, developers had to employ several tricky ways such as Ajax polling, Comet (A.K.A Ajax push, Full Duplex Ajax, HTTP Streaming, etc.), those technologies either periodically fire HTTP requests to server or hold the HTTP connection with server for a long time, which "contain lots of additional, unnecessary header data and introduce latency" and resulted in "an outrageously high price tag". websocket.org explained the problems exhaustively, compared the performance of Ajax polling and WebSocket in detail, built up two simple web pages, one periodically communicated with server using traditional HTTP and the other used HTML5 WebSocket, in the testing each HTTP request/response header is approximate 871 byte, while data length of WebSocket connection is much shorter: 2 bytes after connection established, as the transfer count getting larger, the result will be:
Traditional HTTP Request
Use case A: 1,000 clients polling every second: Network throughput is (871 x 1,000) = 871,000 bytes = 6,968,000 bits per second (6.6 Mbps)
Use case B: 10,000 clients polling every second: Network throughput is (871 x 10,000) = 8,710,000 bytes = 69,680,000 bits per second (66 Mbps)
Use case C: 100,000 clients polling every 1 second: Network throughput is (871 x 100,000) = 87,100,000 bytes = 696,800,000 bits per second (665 Mbps)
HTML5 WebSocket
Use case A: 1,000 clients receive 1 message per second: Network throughput is (2 x 1,000) = 2,000 bytes = 16,000 bits per second (0.015 Mbps)
Use case B: 10,000 clients receive 1 message per second: Network throughput is (2 x 10,000) = 20,000 bytes = 160,000 bits per second (0.153 Kbps)
Use case C: 100,000 clients receive 1 message per second: Network throughput is (2 x 100,000) = 200,000 bytes = 1,600,000 bits per second (1.526 Kbps)
Finally a more readable diagram:
"HTML5 Web Sockets can provide a 500:1 or — depending on the size of the HTTP headers — even a 1000:1 reduction in unnecessary HTTP header traffic and 3:1 reduction in latency". --WebSocket.org
WebSocket In Essence
The motivation of creating WebSocket is to replace polling and long polling(Comet), and endow HTML5 web application the ability of real-time communication. Browser based web application can fire WebSocket connection request through JavaScript API, and then transfer data with server over only one TCP connection.
This is achieved by the new protocol - The WebSocket Protocol, which is essentially an independent TCP-based protocol. To establish a WebSocket connection client/browser forms an HTTP request with "Upgrade: WebSocket" header which indicates a protocol upgrade request, and the handshake key(s) will be interpreted by HTTP servers and handshake response will be returned (the detailed handshake mechanism will be described below), afterwards the connection is established (figuratively speaking, the 'sockets' have been plugged in at both client and server ends), both sides can transfer/receive data independently, no more redundant header information, and the connection won't be closed until one side sends close signal, that's why WebSocket is bidirectional and full duplex, in additional, comparing the request/response paradigm of HTTP, WebSocket layers a framing mechanism on top on TCP, each data frame is minimally just 2 bytes.
Now it is time for us to delve deep into this protocol, let's start with WebSocket version draft-hixie-thewebsocketprotocol-76 which is now supported by browsers (Chrome 6+, Firefox 4+, Opera 11) and many WebSocket servers (please refer to Browser/Server Support section below for details). A typical WebSocket request/response example is shown below:
The entire process could be described as: the client raise a "special" HTTP request which request "Upgrade" connecting protocol to "WebSocket", on domain "example.com" with path "/demo", with three "handshake" fields: Sec-WebSocket-Key1, Sec-WebSocket-Key2 and 8 bytes ({^n:ds[4U}) after the fields are random tokens which the WebSocket server uses to construct a 16-byte security hash at the end of its handshake to prove that it has read the client's handshake.
Since WebSocket protocol is NOT finalized and is being improved and standardized by IETF Hypertext Bidirectional (HyBi) Working Group, at the time I wrote this article, the latest WebSocket version is "draft-ietf-hybi-thewebsocketprotocol-08" lasted updated by Ian Fette on June 7, 2011, in which both request/response headers are changed comparing to version 76, the handshake mechanism was changed as well, Sec-WebSocket-Key1 and Sec-WebSocket-Key1 are replaced with one Sec-WebSocket-Key.
WebSocket request/response in the latest draft-ietf-hybi-thewebsocketprotocol-08:
The Sec-WebSocket-Key is a base64 encoded randomly generated 16-byte value, in the case above it is "WebSocket rocks!", the server reads the key, concats with a magic GUID "258EAFA5-E914-47DA-95CA-C5AB0DC85B11", to "V2ViU29ja2V0IHJvY2tzIQ==258EAFA5-E914-47DA-95CA-C5AB0DC85B11", then compute its SHA1 hash, get result "540b8681a34307fade550a467c317c297799c79a", finally based64 encodes the hash and append the value to header "Sec-WebSocket-Accept".
I've written C# code below to demonstrate how to compute the Sec-WebSocket-Accept conforming to draft-ietf-hybi-thewebsocketprotocol-08:
/// <summary> /// Computes server security hash for HTML5 WebSocket, handshake mechanism /// based on draft-ietf-hybi-thewebsocketprotocol-08 /// http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-08#page-27 /// </summary> /// <param name="secWebSocketKey">The handshake key "Sec-WebSocket-Key" from client.</param> /// <returns>The computed security hash to fill "Sec-WebSocket-Accept" header.</returns> publicstaticStringComputeWebSocketHandshakeSecurityHash08(StringsecWebSocketKey) { constStringMagicKEY="258EAFA5-E914-47DA-95CA-C5AB0DC85B11"; StringsecWebSocketAccept=String.Empty;
// 1. Combine the request Sec-WebSocket-Key with magic key. Stringret=secWebSocketKey+MagicKEY;
// 2. Compute the SHA1 hash SHA1sha=newSHA1CryptoServiceProvider(); byte[]sha1Hash=sha.ComputeHash(Encoding.UTF8.GetBytes(ret));
// 3. Base64 encode the hash secWebSocketAccept=Convert.ToBase64String(sha1Hash);
Please note draft-ietf-hybi-thewebsocketprotocol-08 is NOT supported by any common web browsers since it was just published on June 7, 2011. However, we can learn it so that we know what is happening now with WebSocket and where it's going in the future.
Experimental Demos
So far there are already many experimental WebSocket Demos built based on either draft-hixie-thewebsocketprotocol-75 or draft-hixie-thewebsocketprotocol-76.
http://rumpetroll.com/ Play a tadpole in canvas and can chat with other tadpoles, tadpoles location and char messages could be seen by everyone in real-time.
http://kaazing.me/ Refresh stock, weather, news and twits in real-time powered by Kaazing's real-time message delivery network solution.
Mr. Doob's Multiuser Sketchpad In this multi-user <canvas> drawing application, Web Sockets are used to pass along the coordinates of the lines that other users draw back to each client as they happen.
WebSocket is not only designed for browser/server communication, client application can also use it. However, I guess browser will still be the major platform for WebSocket protocol taken into account the emerging handsets and the coming cloud. At the time I wrote this article, WebSocket protocol version draft-ietf-hybi-thewebsocketprotocol-76 was supported by Safari 5+/Google chrome 6+ (link), Mozilla Firefox 4+ (disabled by default link) and Opera 11 (disabled by default link), IE 9/10 does not support... but on not supported browsers we can use Flash shim/fallback by adopting web-socket-js.
The awesome Can I uses it is maintaining HTML5 new features support in all popular browsers, screenshot below shows WebSocket support:
There are also a number of WebSocket servers available:
http://socket.io - Provides seamless support for a variety of transports (WebSocket, WebSocket over Flash, XHR polling, JSONP polling, etc.) intended for real-time communication developed by Guillermo Rauch
node.ws.js - A simple WebSocket server (support both draft-hixie-thewebsocketprotocol-75 and draft-hixie-thewebsocketprotocol-76) developed based on node.websocket.js.
web-socket-js - HTML5 Web Socket implementation powered by Flash
The url attribute is the WebSocket server Url, the protocol is usually "ws" (for unencrypted plain text WebSocket) or "wss" (for secure WebSocket), send method sends data to server after connected, close is to send close signal, besides there are four important events: onopen, onmessage, onerror and onclose, I borrowed a nice picture below from nettuts.
onopen: When a socket has opened, i.e. after TCP three-way handshake and WebSocket handshake.
onmessage: When a message has been received from WebSocket server.
onerror: Triggered when error occurred.
onclose: When a socket has been closed.
JavaScript code below is to setup WebSocket connection and retrieve data:
Estimating user story effort by using Planning Poker Cards is well-known and widely used in Agile/Scrum development, Program Manager/Scrum Master prepare user stories beforehand, hold meeting with stake holders and have them play poker to represent one's estimation on each story, the higher the card value is, the harder to implement, on the contrary, the lower the value is, the easier to implement, "0" indicates "no effort" or "has been done", "?" indicates "mission impossible" or "unclear requirement".
Actually there is a website - http://pokerplanning.com does the exact work described above, my co-workers and I used it for several times, however, we found it is getting slower and slower as more team members joining the game or after several rounds of voting, we did experience the worst result: no one can vote anymore because everyone's voting page got stucked. I strongly suspect the major reason for this is its Ajax Polling strategy in order to ensure everyone got real-time voting status. By tracking its network activities I guess I was right.
I believe HTML5 WebSocket will solve the problem! So I developed a simple demo (I named it Team Poker) which currently only has limited and basic functionalities described as below:
User can login to poker room after inputting his/her nickname.
Everyone gets notified when one user players a poker.
Everyone gets notified when new player joins.
Newly joined player can see the current participants and voting status.
The login screenshot for requirement #1:
New participant(s), new voted poker(s) status update screenshot for requirement #2 and #3 (please click on the image to enlarge):
Newly joined participant(s) can see current game status, story #4:
Please note the Team Poker demo is concentrated on demonstrating the power of WebSocket, in real world, player shouldn't see the poker values played by other players, and there is functionalities like moderator customizing user stories, storing game status on server side, and there is no fancy UI/animation. However, I've share all the source code at the beginning of this article, in additional, I've uploaded the source code on github: https://github.com/WayneYe/TeamPoker, wish some people make it better and productive, will you fork it with me? Dear reader:).
Ok, now coding time, since all clients need to get notified about other client's changes (new player joining or new poker played), in additional, new joint player needs to know current status, I defined two communication contracts:
ClientMessage indicates message sent from client, contains a Type property reflects enumeration class MessageType - NewParticipaint, NewVoteInfo, and a Data property to store data.
ServerStatus, stores current playing players as well as current voting status - a hashtable [{PlayerName}, {VoteValue}], broadcast to all clients once receiving new client message.
On client side, a WebSocket connection will be established after user clicking "Login" button, the nickname will be sent to WebSocket server running on nodejs, the kernal client code is shown below:
TeamPoker.connectToWsServer=function () { // Init Web Socket connect varWSURI="ws://192.168.1.6:8888"; TeamPoker.WsClient=newWebSocket(WSURI);
TeamPoker.WsClient.send(JSON.stringify(joinGameMsg)); } TeamPoker.updateGameStatus=function (data) { // Format/fill the data from server side to HTML }
On server side, one important task is to maintain all active client WebSocket connections so that it can "broadcast" messages to every client, and remove the closed client to avoid sending message to "dead" client. Other than this, the logic is very simple, validate message type sent from client, update players/vote status repository and then broadcast to all client:
/* WebSocket server based on https://github.com/ncr/node.ws.js Written By Wayne Ye 6/4/2011 http://wayneye.com */
varsys=require("sys"), ws=require("./ws");
varclients=[],players=[],voteStatus=[];
ws.createServer(function (websocket) { websocket.addListener("connect",function (resource) { // emitted after handshake sys.debug("Client connected on path: "+resource);
// # Add to our list of clients clients.push(websocket); }).addListener("data",function (data) { varclinetMsg=JSON.parse(data);
// Notify all clients except the one just sent data varserverStatus=newServerStatus(); serverStatus.Players=players; serverStatus.VoteStatus=voteStatus;
varsrvMsgData=JSON.stringify(serverStatus);
sys.debug('Broadcast server status to all clients: '+srvMsgData); for (vari=0; i<clients.length; i++) clients[i].write(srvMsgData); }).addListener("close",function () { // emitted when server or client closes connection
for (vari=0; i<clients.length; i++) { // # Remove from our connections list so we don't send // # to a dead socket if (clients[i]==websocket) { sys.debug("close with client: "+websocket); clients.splice(i); break; } } }); }).listen(8888);
Instead of spending any cent in the money pit casinos, I’d chose to do something more meaningful and exciting :)
SkyJump Las Vegas
SkyJump Las Vegas holds the Guinness World Record for highest commercial decelerator descent with an official height of 829 ft (253 m) and is located at Stratosphere Las Vegas. As part of its grand opening event, Las Vegas Mayor Oscar Goodman presented a written proclamation deeming April 20, 2010 as SkyJump Day in Las Vegas.
– Wikipedia (https://en.wikipedia.org/wiki/SkyJump_Las_Vegas)
This is my second time attending AWS re-Invent, last time was year 2017, the purpose of this blog is to summarize what I’ve learned, impressed and entertained
At reinvent logo
Crazy crowd at midnight madness party
Overall during my week staying my daily schedule was listening to sessions mixed with slack/emails during the daylight, and party/social in the evening time.
All the keynotes/sessions have been uploaded here: https://reinvent.awsevents.com/learn/keynotes/ and all announcements were summarized at this page, my personal top 5 thrilled announcement...
WHEN YOU ARE OLD
When you are old and grey and full of sleep,
And nodding by the fire, take down this book,
And slowly read, and dream of the soft look
Your eyes had once, and of their shadows deep;
How many loved your moments of glad grace,
And loved your beauty with love false or true,
But one man loved the pilgrim soul in you,
And loved the sorrows of your changing face;
And bending down beside the glowing bars,
Murmur, a little sadly, how Love fled
And paced upon the mountains overhead
And hid his face amid a crowd of stars.
– By William Butler Yeats
当你年老时
...
Background
I was planning to attend #adskfootball2019 in Dublin, Ireland from Jun 14th to Jun 16th, 2019:
Autodesk Football Tournament 2019
In order to attend, I needed applying Ireland VISA, and since I was planning to fly back to Shanghai later in June, so I thought I could just fly from SF to Dublin, then Dublin to Shanghai, and soon I realized that there is no direct flight from DUB to PVG, so a practical (and also very fun and economic) option is to fly from Dublin to London, and then London to Shanghai, bingo ^_^
Leave a comment