One day when I was surfing cat videos professional relevant videos on Youtube I noticed a red progress-indicator:

My first thought was – I want this in my apps. How did they do it?
If you lower the bandwidth a pattern emerges:

Aha! When you click on a video, Youtube will start a request to fetch informations about it, animate the bar to 60% where it waits until the call is completed and finally animates it to 100%.

Utter deception but without doubt a well thought-out solution. As long as you are on a sufficiently fast connection and the amount of data that needs to be transferred is limited the illusion is complete.

It has to be said that even if Youtube cheats a little it is a much better solution than those spinners you see on the majority of sites with asynchronous requests today:

It gives no sense of progress and no indication if the transfer has stopped. I’ve also experienced many sites where errors aren’t handled correctly and you end up with a eternal spinner – or at least until you loose patience and refresh the page.

Dan Saffer expresses it in simple terms in the book Designing Gestural Interfaces: Touchscreens and Interactive Devices

Progress bars are an excellent example of responsive feedback: they don’t decrease waiting time, but they make it seem as though they do. They’re responsive.

With the very diverse connection speeds we have today I’d say that the need is even greater – The Youtube example from before might hit the sweet spot on an average connection, but If you are sitting away from high speed connectivity maybe on a mobile connection with Edge (nevermind that you probably cannot see the video itself) the wait can easily outweigh your patience.

As Jakob Nielsen writes in Usability Engineering; Feedback is important, especially if the response time varies

10 seconds is about the limit for keeping the user’s attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.

Requirements for the solution

There are many ways to add continuous feedback but each come with their own limitations. To be able to use the solution in most problem areas we need to set some requirements:

The solution should:

• Be integratable into existing solutions without too much extra work and (almost) without changes to the server.
• Should work across domains, when the client lives on one domain and communicates with the server on a different. (Also called Cross Origin Resource Sharing or CORS).
• Work in all browsers.
• Be able to send a considerable amount of data both back and forth.

 

The first attempts

When communicating with a server from JavaScript, it is under normal circumstances a request that is started and a little while later you get a status indicating whether the call was a success or not. So no continuous feedback. The naive solution could be to make multiple single requests but the overhead of a request is relatively expensive so the combined overhead would be too big.

If you look at HTTP/1.1 there is a possibility to split a response from the server in multiple parts – the method is called chunked http:

If we send a agreed number of chunks we are able to tell the user how far the request has come.

 

Traditional AJAX call

Traditional AJAX calls – typically executed via XMLHttpRequest generally don’t give any frequent feedback and if you want to execute it as a CORS-call you need to make changes to the server and it will give an extra request if the call is with authorization. The support in older browsers is limited.

 

JSONP

JSONP is an old technique to circumvent the CORS problems with AJAX. A call where you basically add a script tag that lives on the remote server that will then be executed within the scope of the current page.

In a weak moment I tried to implement chunked http in a JSONP call – multiple JavaScript methods in one script that would then be executed continuously. Of course it didn’t work – the browser won’t execute any code until the JavaScript file has been fully loaded.

 

Other technologies

I also looked at Server Sent Events, but thats not supported by Internet Explorer.
WebSockets requires a greater change on the server side and it also gives some challenges with security, cookies etc. when we are outside the traditional http-model.

The final solution

HiddenFrame is a technique where you create a hidden iframe and let it fetch a lump of HTML from the server. If there is any script-tags in this lump, they will be executed as the browser meets each end-tag. So there we have a potential solution.

Sending data is no problem either because we can start the fetch itself by executing a form-post.

And where does JavaScript Promises fit into all this?

Well to get a good API I’ve used a Promise-implementation that offers continuous feedback via the progress-method and handles any success and failure-scenarios:

new LittleConvoy.Client('HiddenFrame').send({ url: 'http://example' }, { name: 'Ford'})
    .progressed(function (progress) {
      ... progress contains percentile progress and will be called 10 times
    })
    .then(function (data) {
      ... the call was a success and data contains the result
    })
    .catch(function (message) {
       ... the call failed and message contains the error message object
    });

On the server side a library is added, currently only available for Microsoft ASP.NET MVC. The JSON producing methods that you already have are just decorated with a attribute that makes sure that everything works:

public class DemoController : Controller
{
    [LittleConvoyAction(StartPercent = 40)]
    public ActionResult Echo(object source)
    {
      return new JsonResult {Data = source };
    }
}

 

Demo or it didn’t happen

• A small demo is available here.
• The code is available at GitHub
• The library can be installed via the .NET package manager NuGet as LittleConvoy.

 

The future

• The transport layer itself is separate from the client so extra methods can be added, adding traditional AJAX and WebSockets would be an obvious choice in the future if it can be done with too many changes on the server side.
• Some Promise implementations offer cancellation, it would be great if you could cancel a call.
• Sending data gives no feedback – it would be great if that was somehow possible.

 

Related work

• Comet is a collection of technologies that offers push for the browser and it also contains a implementation of iframe-communication but is targeted permanent communication channels.
• SignalR is like Comet build for push and permanent communication channels
• Socket.IO is an abstraction over WebSockets that contains fallback to iframe-communication but more targeted Node.js.

This post is also available in Danish at QED.dk