Tagged: SPOF

“Grunt” your way to frontend performance optimization

Performance optimization has been more than ever in the spotlight of web developers, especially for mobile web developers who have to understand and know by heart the challenges and constraints of mobile devices: these devices run off a battery that e.g. drains faster if performance is not taken seriously. The devices are powered by smaller CPUs than desktop devices. Unknown factors like latency and network connectivity challenge developers to build slim, light-weight and fast websites. Data plans still remain expensive and inconsiderate use of served data by web developers should not be ignored.

Clearly, performance is (and should) not (be) an after-thought anymore. When web developers create websites, performance can influence the success or failure of a web product. We’ve been hearing from leading performance advocates like Ilya Grigorik that speed is a feature and should not only be thought of just before a product hits production but rather as an essential part of the web product development cycle.

For example, instead of minifying and concatenating CSS and JavaScript files manually, tools and processes are out there that can help and put these performance tasks into an automated workflow, and more importantly right from the beginning of a product development cycle.

I’ve been using Maven to run most of the automated performance optimization at work, however I’ve been always interested in using Grunt for the same purpose. Grunt is a task runner, created for web products, based on JavaScript that can be leveraged to make performance part of the deployment process.

In today’s blog post I will be sharing some of the plugins for Grunt that can be used to speed up and automate performance optimization. At the end of the blog post, I will present performance results that will show that frontend optimization (FEO) can be fun and easily be automated to cut page load time.

Google’s “Make the Web Fast” team recommends frontend best practices as well as Steve Souder’s “High Performance Web Sites” outlines rules that can be applied for FEO. I decided to pick two of Steve’s rules “Make Fewer HTTP Requests” and “Minify JavaScript” (and CSS, HTML) by using Grunt plugins that can help automate those specific rules. So here it goes.

Note: The post assumes that you’ve worked with Grunt before and know how it’s been installed, and how to install plugins (I won’t go into details, however links at the bottom will help you)

“Let’s grunt it up”

(All plugin headings in this post are clickable links to their appropriate pages)

grunt-montage

Montage helps you sprite images to reduce HTTP requests. You will need ImageMagick to be installed. Alternatively, you can also try out grunt-spritefiles.

grunt-usemin

This plugin is useful when you want to develop and debug a version of your site that doesn’t use the minified and concatenated version of your JavaScript or CSS files. A comment blocks is wrapped around your JavaScript and CSS assets that will be concatenated to your destination after deployment.

<!-- build:js js/magic.min.js -->
<script src="js/1.js"></script>
<script src="js/2.js"></script>
<script src="js/3.js"></script>
<script src="js/4.js"></script>
<!-- endbuild -->

will become

<script src="js/magic.min.js"></script>

You can use grunt-processhtml instead.

grunt-closure-compiler

Alternatively you could use grunt-closure-compiler instead of combining concat and uglify and cssmin for JavaScript and CSS files.

grunt-contrib-uglify

Uglify and concat go almost hand in hand and should be used together, the concat plugin first makes sure to combine all defined JavaScript files. Uglify only works on JavaScript files. It minifies all code in a one line block of code. Use cssmin for CSS files.

grunt-contrib-cssmin

Same logic and idea than uglify, once your CSS files are all concatenated, use cssmin to shrink several lines of CSS code into one single one.

grunt-contrib-concat

Combine CSS and JavaScript files with this plugin, it allows you to reduce your HTTP requests for each and every file to just one combined file.

grunt-contrib-imagemin

imagemin minifies JPG and PNG images. It’s a handy Grunt plugin if you don’t know if the assets you got handed from your designer (or yourself) are optimized for web yet. By using this tool, you have the piece of mind that you use image files in an efficient way. Alternatively, you could use grunt-smushit, it’s based on Yahoo’s great smushit tool that is available in the YSlow plugin for several browsers.

grunt-image-embed

This plugin encodes images as base64 and leverages the technique of data URIs for images, something that can be used inline with CSS to reduce HTTP requests and hence reduce page load time. I’ve written a blog post where this is explained in more detail.

grunt-htmlcompressor

This plugin is using htmlcompressor tool to minify and compress HTML files. The options parameter is handy to tweak your compression, my example uses compressJS and preserveServerScript to also compress inline JavaScript and server script tags in case I wanted to include some SSI code.

spofcheck

Use grunt-exec to run the SPOFcheck. An excellent tool to identify bad 3rd party scripts includes, developed by the eBay team. I didn’t include the scripts asynchronously, hence SPOFcheck complaints to avoid SPOF.

“It’s Magic” Sample Page

I created a simple page themed “It’s magic” where I applied all mentioned plugins. You can find the files including Gruntfile.js here. Please note, I intentionally didn’t put a lot of effort into the styling (It is supposed to look as simple and cheesy as it feels to you)

In a nutshell, the page has a logo, uses JQuery from the Google CDN, includes a simple JQuery gallery with previous and next buttons. Simple JavaScript and CSS files are being used.

“without/magic.html”

The logo is a png logo, the images are not optimized. There are several CSS and JavaScript files that are all individual being included, not minified nor concatenated.

“with/magic.html”

This file is the one that Grunt will create for you. Visually, the file doesn’t look that different, besides the fact that the title has changed….see yourself

Below are screenshots of the two pages (and links) side by side, the one on the left before Grunt tasks were applied. The right one shows the page after Grunt tasks were applied. For the user they both look the same (except for the heading).

without/magic.html with/magic.html
No magic here!

Can you spot the differences?

  1. The logo was being transformed into a data URI
  2. The title has changed from It’s not magic to It’s magic
  3. The local CSS and JavaScript files were being minified and concatenated
  4. The HTML was being compressed, comments were taken out automatically
  5. The next and previous buttons were converted to a sprite file
  6. On build, SPOFcheck was applied and gave us the following warnings so we could address possible SPOF issues

Screen shot 2013-08-02 at 7.25.05 PM

Let’s take a look under the hood

Here are the waterfalls for both versions:

Without magic
notmagic-waterfall

With magicmagic

WebpageTest Results

I ran WebpageTest for both files with 9 runs for IE8 with a DSL connection to retrieve the median. Here are the performance results:

  1. Without magic results
  2. With magic results
  • HTTP requests dropped by ~48%
  • Page load time (PLT) dropped by ~10%
  • File sizes dropped by ~10%

Even if those numbers are not that high (mostly due to the simplicity of the experiment), it shows that Grunt can help you automatically optimize your deployment process.

As you can tell by the sample code, there are many mix and match options available, depending on the magnitude and granularity of your page structure. Nevertheless, this little sample shows how to use Grunt to optimize performance and to illustrate what is possible. Feel free to use the code as a starting point, and tweak or customize it to your likings.

General references and info to get you started with Grunt

Simulating Frontend SPOF – The day a tiny 3rd party script almost slowed down the entire Internet

How realistic is it really that a script that you didn’t even write could dramatically slow down your site and other major sites as well? Keep reading….scripts can slow down sites and it hurts to watch!

I watched the Fluent talk by Steve Souders from 2012 about High Performance Snippets (must-watch for all SPOF fans) and got inspired to test out how an “innocent” 3rd party script (btw. I call them 3rd party monsters), not loaded properly could result in a single point of failure (SPOF) and to make a site very slooooooow to load.

Developers are always proud and optimistic about their code, and when it comes to including 3rd party scripts, basically code they don’t usually touch, they assume those 3rd party providers like Google never go down, a non-responsive ad server like DoubleClick won’t hurt or Twitter won’t have server failures. 3rd party script developers try their best to make it easy and painless for us to include their high performing scripts into our sites. That’s a fact. True but also not true. They can only do so much. If you don’t properly include the script on your end and their service goes down, their high performing code won’t be able to help you at all. The rule of thumb is to include those scripts asynchronously. That way you make sure that your content won’t be blocked from rendering in case the 3rd party service is down.

However, scripts that use document.write can’t be loaded asynchronously (unfortunately). Read more about this in the great Krux post and some of Steve Souders’ posts.

It’s kind of like the elephant in the room to me; you pray e.g. that Twitter doesn’t go down meanwhile you are too afraid to test it out or are over-confident that it won’t break your page or you basically don’t really know how you would test this scenario in the first place. Am I right? Well, what if you could run a quick test on a web site and pretend all of their 3rd party scripts and providers were down. Let’s play the “3rd party scripts game“: would your web site still render…how confident are you?

Simulating SPOF – Slow down your own site until it really hurts

Are you ready for this? First, edit your hosts file to point to a blackhole IP address for simulation (I used the blackhole IP address Steve shared in his talk on slide 9).

sudo vi /etc/hosts

While setting up my test, I don’t want to play the really bad gal (yet) and assume all 3rd party providers were down. I’d like to start with the simplest but yet most used and harmful domain ads.doubleclick.net. A lot of web sites include ads and use DoubleClick.

So let’s use this domain for our blackhole test. By all means, you can add more 3rd party scripts to your hosts file.

// add this line to your hosts file
72.66.115.13 ads.doubleclick.net

Once you’ve updated your host file, remember to flush your DNS cache after.

dscacheutil -flushcache

Now, open your browser (with cache disabled so your browser is not using any DoublClick scripts from the cache). Type in your site’s URL and be prepared for the worst. How long will it take for the website to load?

That’s a very easy (scary) and quick way of evaluating what is on your critical rendering path and obviously (now) what should not be on it anymore!

I ran this test on our site and let me tell you, it hurt. Period. It took almost 1.5 min for cbc.ca to display useful content faking that DoubleClick was down. The browser finally gave up.

Aborting

I wasn’t ready to stop the game. I wondered if it’s just our site that doesn’t properly handle the outage of one single domain such as ads.doubleclick.net. So I continued and tried the following random websites and measured the time it took so see useful content on those.

URL Time past to see useful content
http://www.people.com ~4.5 mins
http://www.bbc.com ~2.5 mins
http://www.amazon.com Fine, didn’t seem to use DoubleClick
http://www.cnn.com Fine, they seemed to be doing the proper handling
http://www.facebook.com Surprise, surprise Facebook doesn’t use DoubleClick. They use their own, so no real delay here.

 

If you don’t want to edit your hosts file and want to get more concrete waterfall and timing information as well as video captioning, try out what Steve Souders suggested in his Fluent talk by using the scripts (now SPOF) box at webpagetest.org to include DNS changes. The results will give you great details on how the website performed, with and without SPOF.

SPOF doubleclick

Note: I’ve tried WebPagetest SPOF myself and didn’t notice a big difference between non-SPOF and SPOF version; my suspicion is that WebPagetest might not be using empty cache for SPOFs setting. The tests I ran manually on my local machine showed more visible negative impact of the SPOFs (I shall confirm this).

3rd party scripts are everywhere

It was verified last month that 18% of the world’s top 300K URLs load jQuery from Google hosted libraries. So that means in theory if that service goes down and a web site uses JQuery from ajax.googleapis.com (and doesn’t have a fallback), the site might not work at all. Isn’t that scary? If you develop for a web site that already uses a CDN, don’t use Google’s CDN for scripts like JQuery. Avoid those 3rd party dependencies as much as possible.

I ran two queries on my local HTTP Archive database (dump from March 2013) and followed the same filter that Steve Souders used above. I restricted the query to only look at 292, 297 distinct URLs from the March 1 2013 crawl (with their respective unique pageid’s). I wanted to see how many of the top 300K URLs use Twitter widgets and any sort of Facebook scripts (without a distinction if they were loaded synchronously or asynchronously).

Twitter

Twitter

13% of the Top 300K URLs include Twitter scripts somewhere on their page.

Facebook

Facebook

29% of the Top 300K URLs include Facebooks scripts somewhere on their page.

Feel free to extend this exercise to include more 3rd party domains.

Cached 3rd party scripts

You can’t really rely much on the cache settings of your 3rd party scripts to ignore their outage if it happens for less than a few hours. 3rd party providers tend to set a very low cache time on their scripts to make it flexible for them to change the file frequently.

That setup plays against you in the case where you don’t load 3rd party scripts asynchronously. For example Twitter’s widget.js has a cache time set to 30 mins (only). I wonder what change could be so important for Twitter that can’t wait for more than 30min to be loaded on sites consuming this widget.js file.

So imagine the following: You go to a site with the Twitter widget loaded synchronously at the top of the page (bad!) at 9 AM (getting the latest, freshest version of widget.js). Twitter goes down at 9:10 AM. You go back to the site you visited at 9 AM, now at 9:15 AM, everything is still fine, you won’t see any problems because you are getting the cached Twitter widget script from the browser cache. What if Twitter is still down at 9.40 AM and you visit the same page again, you now are past the cache modified time and your browser will request a new version of the Twitter script, trying to reach the Twitter server that is still down. You are now getting a time out response for the Twitter script that (with the setup described above) will block the page content from rendering. Bottom line, you wouldn’t be able to see any content until Twitter is back up (and the cache has expired). It’s easy to check those cache times yourself, e.g. use Chrome dev tools and check out the response headers from those 3rd party scripts.

The screenshot below shows Twitter and Facebook’s cache-control settings:

TwitterFacebook

Conclusion

In order to really focus on your site’s performance, you need to isolate (potential bad) performance of 3rd party monsters (the ones that you decided to invite to your site). Don’t make your users wait for your own content if a 3rd party provider is down.

References