Loading blog entries.. loading

HTML5 Web Storage in Essence

Saturday, February 26, 2011 11:54 AM
Written by Wayne Ye
Font Size: S  M  L 

Background

Web Storage (also named Dom storage) is a brand new mechanism of storing temporary/permanent data introduced in HTML5 to replace cookie, it contains two new DOM objects: sessionStoragy and localStorage), in Web Storage Spec, Ian Hickson documented that Web Storage are intended to solve two cases that "not handled well by Cookie", they are:

  1. The first is designed for scenarios where the user is carrying out a single transaction, but could be carrying out multiple transactions in different windows at the same time.
  2. The second storage mechanism is designed for storage that spans multiple windows, and lasts beyond the current session. In particular, Web applications may wish to store megabytes of user data, such as entire user-authored documents or a user's mailbox, on the client side for performance reasons.

sessionStorage and localStorage are introduced by W3 in HTML5 to solve problems above, let's begin with a review of how Cookie currently works.


Current HTTP State Storage – Cookie

Mechanism

An HTTP Cookie was a set of key/value pairs which an HTTP web server requests a concrete client browser (i.e. User Agent - UA) to store on client's hard disk, when the UA request to the web server further, UA will wrap all the cookie data within the HTTP request header as long as following conditions are all satisfied:

  1. The cookie data has not expired.
  2. The requested domain is under exactly the same domain when cookie was stored.
  3. The requested path under the domain is the same one where cookie was stored.

Once the cookie expires, browser will delete it from local hard disk.

A common cookie example:

  1. An end user types www.foo.net in a browser's navigation bar and hit enter.
  2. www.foo.net returns HTTP status 200 message below (In the Green part, web server expected client browser to store two Cookie key/value pairs: WayneKey1=value1;WayneKey2=Value2, they could be accessible from the entire domain, and they will expire after 1 year):

    HTTP/1.1 200 OK
    Content-type: text/html
    Set-Cookie: WayneKey1=value1;WayneKey2=Value2;domain=www.foo.net; expires=Tue, 24-Feb-2012 07:13:22 GMT; path=/; HttpOnly
     
    (Response HTTP body)

  3. Browser received the HTTP response and stored the cookie key/values in plaint text into user's hard disk.
  4. User did some interactions and requested back www.foo.net, browser will wrap the cookie data into request HTTP header.

    GET /Index.aspx HTTP 1.1
    HOST: www.foo.net

    Accept: *.*
    Cookie: WayneKey1=value1;WayneKey2=Value2;

    (Request HTTP body)

Notes: in #4 above, if client browser does not support cookie or cookie is disabled by user , a fallback mechanism might be implemented by the HTTP Web server, which can usually store cookie data within the URL (for an instance example, ASP.NET can support storing cookie in URL by setting cookieless="UseUri" in web.config, refer: Cookieless ASP.NET), or alternatively, the Web server won't allow client to use its features.

Based on a server allocated "Session ID" storing in client cookie, Session object is widely used in all popular web servers to store temporary data on server side (could be memory, database or a dedicated session storage server and so on), in case client disabled cookie, web server might adopt the same strategy mentioned above (storing cookie in URL), or instead, they do not allow client to use its features except client gets cookie enabled.

Situation

Nowadays countless websites (for example Google, Facebook, Amazon, New York Times) relies on HTTP cookie to store data such as user preference, login information, shopping cart and so on. If you disable cookie in your browser and try to access many extremely popular web applications (like Facebook, Twitter, Gmail, Amazon and so on) today , you probably see screenshots below:

In one word, cookie enabled is a mandatory condition to use these web giants.

Cookie Drawbacks

I investigated and summarized a number of drawbacks of Cookie listed below (two W3C points mentioned at the beginning are included):

  • Size and count limit
    Most browsers limited cookie size to 4096 bytes and 300 maximum cookie count within a domain. IETF recommended limitation standard, refer: http://www.ietf.org/rfc/rfc2109.txt section 6.3.
  • Performance hit
    If a website uses cookie, then every HTTP request/response between server/browser must include all cookie key/values pairs in the HTTP header.
  • Security
    Cookie is stored in user's local hard disk in plaint text. If developers didn't deal with this appropriate, cookie/session hijacking could possibly happen.
  • Ability to store data separately for more than one instance of same web application
    Cookie is not easy to handle one use case: separate instances of the same web application to run in different windows without interfering with each other (sessionStorage is going to well-support that, I will describe it later in this post).

Web Storage mechanism

Unlike cookie which passed between server/client and could be accessed by both of them, sessionStorage/localStorage are 100% stored in the client by a concrete browser, sessionStorage stores temporarily data in one HTTP session, localStorage stores permanent data into client hard disk. The advantage is obvious:

  • Data won't be passed through HTTP request/response, bandwidth will be saved.
  • There will be no 4KB limitation, web site has much more flexibility to store large data in client.

    Notes: W3C "recommended 5 megabytes localStorage size limitation per domain", and "welcome feedback", this is much larger than 4KB limitation in cookie.
  • Now that data won't be passed through network, this will be relatively more secure.
  • Considering further according to #3, a number of existing HTTPs connection in theory could use plaint HTTP by adopting Web Storage, because no need to encrypt the data, the data is stored in client side.  Since HTTPs usually has only 10% performance comparing with HTTP,  eventually either performance will be improved or cost is saved (procuring cheaper server hardware) .

In my humbly opinion, Web Storage is one the greatest feature in HTML5, and I believe it will lead an HTTP web state storage revolution in the near future! Why? Considering several stories listed below, I guess they will very possibly happen soon!

 

  1. Bandwidth save. You have an Email account of a popular Email service web application, and you use it daily, at very first time of your visit in a concrete browser. It stores your 300+ contacts list as well as your newest 20 email threads (first page) into your browser's local storage, and the data takes approximate 8 kilobytes in total. In future visits, this 8 kilobytes will not be transferred from the Email server.
  2. No download time = Better UX. You like watching videos from a popular video web site, which records video list (by default 10 latest watched videos) you've watched as well as indexed real video stream data. Every time you visit the web site, you have a ability to view your watching history and most important, you can re-watch any video in the list in a second – from your local storage.
  3. Surfing same web sites using more than one account simultaneously. For some reason, you registered two accounts in one web site, which uses session storage, and you can login with these two account in two browser tabs and interactive in isolation. For example, checking Gmail in two Tabs using two Gmail account at the same time.

Manipulating Storage in JavaScript


The Storage Interface

Both sessionStorage/localStorage are inherited from an interface Storage, it defined in Web Storage Spec:

interface Storage {
  readonly attribute unsigned long length;
  DOMString key(in unsigned long index);
  getter any getItem(in DOMString key);
  setter creator void setItem(in DOMString key, in any value);
  deleter void removeItem(in DOMString key);
  void clear();
};


Basic Operations

First things first, we need check whether a concrete browser supports Web Storage or not, per Wikipedia: Comparison of layout engines (HTML5), IE8+ (IE8 does NOT support Storage Event, while IE9 RC supports), FireFox 3.5+, Safari 4.0+, Chrome 4.0+ all support Web Storage. We can write a  simple JS function to check whether client browser supports Web Storage or not:

function supportsLocalStorage() {
    return ('localStorage' in window) && window['localStorage'] !== null;
}

if(supportsLocalStorage()) {
    // Web Storage invocation here
}
else {
    alert('Sorry, your browser does not support Web Storage');
}

If the browser does support, then invoking sessionStorage and localStorage in JavaScript is fairly easy. Sample code which demonstrates set/get/remove/clear items storage shows below:

// Set/Get locallStorage
localStorage.{key} = strValue;
var localVal = localStorage.{key};

// Set/Get sessionlStorage
sessionStorage.{key} = strValue;
var sessionVal = sessionStorage.{key};

// Remove an storage item
localStorage.removeItem('key')
sessionStorage.removeItem('key')

// Clear storage
localStorage.clear();
sessionStorage.clear();

Please be aware of one thing: both sessionStorage and localStorage can only store "DOM string" value, what if we want to store JavaScript object into them, the answer is definitely JSON string, and someone at StackOverFlow has provided an elegant way to extend Storage.setObject/getObject method by using JavaScript prototype, code shows below:

Storage.prototype.setObject = function (key, value) {
    this.setItem(key, JSON.stringify(value));
}
 
Storage.prototype.getObject = function (key) {
    return this.getItem(key) && JSON.parse(this.getItem(key));
}
 
// By extending Storage's prototype above, just simply add/get JavaScript object into Storage
localStorage.setObject('key',objValue);
localStorage.getObject('key',objValue);
sessionStorage.setObject('key', objValue);
sessionStorage.getObject('key', objValue);

My attached WebStorageDemo shows how to store both DOM String and JSON String into sessionStorage/localStorage, code snippet and screenshot showing below:

Store several fake Shopping Cart items into sessionStorage:

function S4() {
    return (((1 + Math.random()) * 0x10000) | 0).toString(16).substring(1);
}
function Guid() {
    return (S4() + S4() + "-" + S4() + "-" + S4() + "-" + S4() + "-" + S4() + S4() + S4());
}
function ShoppingCartItem() {
    this.ProductID = Guid();
    this.ProductName = '';
    this.CategoryID = -1;
    this.Price = -1.0;
}

// Fake items in shopping cart, and store into sessionstorage
var item1 = new ShoppingCartItem();
item1.ProductName = 'HP WebOS Topaz';
item1.CategoryID = 8;
item1.Price = 849.99;
var item2 = new ShoppingCartItem();
item2.ProductName = 'Apple Ipad II';
item2.CategoryID = 10;
item2.Price = 799.99;

var currentUserShoppingCart = new Array();
currentUserShoppingCart[0] = item1;
currentUserShoppingCart[1] = item2;
sessionStorage.setObject('UserShoppingCart', currentUserShoppingCart);


Delve deep into sessionStorage/localStorage mechanism

As I mentioned at the beginning, both sessionStorage/localStorage are "100% stored in the client by a concrete browser", the difference between them is that sessionStorage is "window/tab isolated" while localStorage is shared cross pages under a domain, in other words, sessionStorage takes care of storing temporary data during an HTTP session life time happened in one browser window/tab, each window/tab can maintain its own session data; While localStorage stores data persists into user's hard disk, all pages under this concrete domain can manipulate the localStorage.

Let's again see a demo, I added a "Counter" which stored natural number into sessionStorage, and a clickable button can increase the Counter, code below:

<input type="button" value="Counter++" onclick="sessionStorage.Counter++;" />

And I open two instance of the page, respectively click "Counter++" on each page, then we will see each of them keeps its own counter:

On the contrary, data stored in localStorage could be accessed by all pages under the same domain, in the demo shown below, initially I inserted a fake "UserProfile" into localStorage, and then open another page which has different path but under the same domain, we will see "UserProfile" stored by first page can be retrieved.

Notes: now that localStorage is shared, it therefore might be manipulated by more than one page simultaneously, W3 emphasized that the browser should implement "Storage mutex" to ensure the Thread safe of localStorage.

By achieving these mechanisms, sessionStorage/localStorage is supposed to replace Cookie with a more straight-forward, effective and secure manner.

 

Storage Event

W3 defines StorageEvent interface as below:

interface StorageEvent : Event {
  readonly attribute DOMString key;
  readonly attribute any oldValue;
  readonly attribute any newValue;
  readonly attribute DOMString url;
  readonly attribute Storage storageArea;
  void initStorageEvent(in DOMString typeArg, in boolean canBubbleArg, in boolean cancelableArg, in DOMString keyArg, in any oldValueArg, in any newValueArg, in DOMString urlArg, in Storage storageAreaArg);
};

Most famous browser (latest version) have implemented StorageEvent, see screenshot below below:

By reading their documents: IE 9, Firefox, Safari, the standard way to subscribe storage event is window.addEventListener('storage', onStorageChanged, false), unfortunately, after trying 2 days, I recognized that it ONLY works in IE 9 RC at this timestamp (2/24/2011), even described by Safari official Dev document, it doesn't work! So I am demonstrating StorageEvent in IE 9 RC, in code below, I subscribes storage event, once sessionStorage/localStorage items got changed, I checks storage's key to determine further reaction:

if (window.addEventListener) { // IE9, FF, Chrome, Safari, Opera
    window.addEventListener('storage', onStorageChanged, false);
}
else if (window.attachEvent) {
    window.attachEvent("onstorage", onStorageChanged); // IE 8
}
else {
    alert('Sorry, your browser does not support DOM event subscribtion!');
    return;
}

function onStorageChanged(e) {
    if (e.key != '') { 
        switch (e.key) {
            case 'Counter': // Counter stored in sessionStorage
                $('spCurCounter').innerHTML = 'Current Session Counter: <b>' + e.newValue + '</b>';
                break;
            case 'UserShoppingCart':
                break;
            case 'UserProfile': // UserProfile stored in localStorage
                break;
            default:
                $('informationBar').innerHTML += 'New localStorage object added, Key: [<span style="color:red">' + e.key + '</span>] Value: [<span style="color:red">' + e.newValue + '</span>].<br />';
                break;
        }
    }
}

The GIF below demonstrates that when I insert a new object into localStorage, onStorageChanged function was triggered on two page instance simultaneously, and displays the new inserted data at the same time:

  

 

Storage Location/Format

Let's finally look into how browser store localStorage: location and data format. Let's take IE9 and Google Chrome on Windows 7 as example, see table below:

Browser localStorage location Data Format
IE 9 RC %userprofile%\AppData\Local\Microsoft\Internet Explorer\DOMStore\ Plaint XML
Chrome %userprofile%\AppData\Local\Google\Chrome\User Data\Default\Local Storage SQLite

IE 9 actually stores localStorage items a simple xml file, below is how the XML stores "UserProfile" object in my demo:

<root>
    <item 
    name="UserProfile" 
    value="{&quot;NickName&quot;:&quot;Wayne&quot;,&quot;EmailAddress&quot;
    :&quot;WebMaster@WayneYe.com&quot;}"
    ltime="3439488560" 
    htime="30135140" />
</root>

Chrome stores data in a SQLite simple database table with two columns: Key and Value, screenshot below:

 

Demo code notes

I've also put the compete Demo on to my blog: http://WayneYe.com/Demo/HTML5/WebStorageDemo.htm

Please note if you test my attached Demo or try any code related with Web Storage in IE 9, please make sure you added X-UA-Capability meta information in your HTML header, or explicitly set both Browse mode/Document mode to "Internet Explorer 9 Standards" in IE Developer tool, otherwise you probably see JS errors since IE 6,7 does NOT recognize session/local Storage.

<meta http-equiv="X-UA-Compatible" content="IE=9" />


Conclusion

Compare to HTTP Cookie, HTML5 Web Storage mechanism provides relatively more convenient, flexibility, secure and faster way to store HTTP state data, it should replace HTTP Cookie gradually, especially nowadays most of popular web browsers had supported it (although there might be several issues such as StorageEvent I mentioned above), they do conform W3C standard to store data in sessionStorage/localStorage. Ian Hickson "expected HTML 5 becomes Candidate Recommendation stage during 2012", let's be expecting and excited to see how things going in the coming future.

 

Further Read

Web Storage Specification
Taking a Dive into HTML5 - Web Storage
Local Storage - Dive Into HTML5
Comparison of layout engines (HTML5)
HTTP_Cookie
HTML5 tests - storage
Introduction to DOM Storage
Safari Developer Library - Key-Value Storage
DOM Storage - MDC Doc Center

 

Also posted at: http://www.codeproject.com/KB/HTML/Web-Storage-In-Essence.aspx

 

Permalink: http://wayneye.com/Blog/HTML5-Web-Storage-In-Essence 6149 Views  5 Comments
Tag: Category:Programming»Web Development»HTML/CSS

 

 


↓Comments↓

  MichaelWednesday, May 25, 2011 4:41 PM
Hi, Thank you for a clear and easy to understand article. One point I would like to make is, surely it's just web traffic that would be reduced rather than storage. I like the idea that when I open my email account it would be faster because all the history is held locally. From a storage (web) point of view though all my email still has to be held there surely so that I can access it from any machine. Still a great read thank you
 
  WayneWednesday, May 25, 2011 4:52 PM
@Michael Thanks a lot for the great feedback:) Web storage is indeed a revolutionary change in HTML5, I guess people might use it in more wisely way in the future, let's see:)
 
  OctaviousSunday, February 5, 2012 7:49 AM
According to your questions, my first impression was that you might haven't had a very clear understanding of cookie, localstorage, HTTP and ASP, let me try to explain them to you in simple sentences, and try to answer your questions. Cookie is essentially key-value pairs stored in client's hard drive, the read/write operations are handled by browser, it can be (most of the time) created/modified/deleted by HTTP response came from server, browser also provided standard JavaScript API to operate. And, cookie MUST be transferred between each HTTP request/response within on domain. localStorage is involved in HTML5, it is NOT created by HTTP response, it is completely controlled by JavaScript code. About ASP, in theory it has nothing to do with cookie/localstorage, in ASP we can generate HTTP headers to tell client browser to control cookie (this can also be done by writing JS code in asp file), however as I mentioned above, localStorage can only be controlled by JS.
 
  OctaviousSunday, February 5, 2012 7:50 AM
@Octavious Your questions: Q: Can you use localStorage likes cookies wherein the stored data can be accessed later on another page A: Yes, as long as the pages are in the same domain (however there is exception: http://www.nczonline.net/blog/2010/09/07/learning-from-xauth-cross-domain-localstorage/). Q:and more specifically, using classic ASP and VBScript A: Yes, as I clarified, localStorage actually has nothing to do with ASP in theory, it is completely client side while ASP is completely server side. Q: Is it possible to display or make use of the localStorage data by passing it as an ASP variable? A: The answer is yes, client JS can read the value from localStorage and send to server side via AJAX. However, I am wondering why you supposed to do so? The target of localStorage is to store completely in client hard drive, so it can be consumed in future client visit, IMHO most of the time server side code needn’t access localstorage values. And a noticeable thing is both cookie
 
  OctaviousSunday, February 5, 2012 7:54 AM
Hi Wayne, I recently read your site documenting the use of localStorage. I currently have several sites - mostly for mobile devices that makes heavy use of cookies and was interested in converting them to localStorage as it sounds more reliable. Can you use localStorage likes cookies wherein the stored data can be accessed later on another page - and more specifically, using classic ASP and VBScript? I noticed that localStorage is written mostly in javascript. We make use of cookies to store user's login values (passwords and a handful of other user's data) as we cannot (due to security restraints of the data being accessed) store these values on our servers and make the comparison. All our site pages are written in classic ASP. Is it possible to display or make use of the localStorage data by passing it as an ASP variable? I can't seem to find any documentations on that online. Or how can I compare localStorage data to SQL data that is retrieved remotely usi
 

Your view point or opinion?
Nickname *
 
Gravatar *
Required (not shown), used only for displaying Gravatar and receiving future notification when new comment(s) posted on this blog.
 
Website/Blog
 
Content *
Current length:     Maximum allowed: charactors