At inclusion

Introduction

This document is a proposition to handle ajaxisation using a declarative way.

To do so, a new part of the URL is introduced: the At Inclusion.
At Inclusion simply describes what to load and where to inject the loaded contents.

You are free (and even encouraged) to ask questions, to make remarks, to share your opinion and fork it via github if you feel the need.

Abstract

Ajaxising a website has several benefits, as well for the webmasters as for their users [1], such as :

1. it reduces the waiting time, since only a part of the document is requested;
2. it makes the interface more responsive, since only a part of the document is changed;
3. it reduces traffic between the server and the client, the linked scripts or CSS files are transferred only at the document load;
4. you can customize what happens when contents are requested, for example by adding animations.

However, nowadays, there are also several main drawbacks to use Ajax :

1. development time and cost are high;
2. without framework, XMLHttpRequest is complex, and requires to know JavaScript;
3. ensuring that the website is indexed is not obvious;
4. some visitors do not have a browser with JavaScript enabled, and would not navigate through the website if the webmaster does not provide a solution;
5. managing the history and binding the URL to the current page is clearly difficult;

At inclusion aims at keeping the benefits of Ajax and at finding a solution to each drawback, by:

1. providing an easy way to request and to inject remote contents;
2. updating the URL and managing history as the user loads contents, so he can bookmark or share the URL of the current page;
3. giving to search engines the ability to index websites using At Inclusion;
4. being compatible with clients having JavaScript disabled;

Definitions

• Inclusion (via URLs): any possible way to include remote contents in the current document via the URL (example: https://twitter.com/#!/w3c);
• At Inclusion: the suggested way to make inclusions;
• At Inclusion pair: a pair of this form : CONTAINER_ID=URL;
• At Inclusion string: a string containing a set of At Inclusion pairs;
• At Inclusion operation: the suggested ways to add, to remove and to replace At Inclusion pairs;
• Operation string: a string containing a set of At Inclusion operations;
• Ajaxisation: in this document, Ajaxisation describes any method to load new contents asynchronously without reloading the whole document.

Objectives

Ease of use

Whereas XMLHttpRequest is a complex object for beginners (who often use libraries instead of it directly), the feature must cover with an easy and rapid way the use case of the injection of remote HTML content, which is one of the most frequently used.
A declarative approach would satisfy that constraint.

Security strength

The feature must prevent the user from creating security issues. The ability to handle inclusions in URLs is not only given to the webmaster, but also to anyone.

If the webmaster has not pointed out this issue, the user can easily create this kind of URL:
http://mysite/mypage.html#SOMEWHERE_UNEXPECTED=/something_I_dont_want_there.html

So the feature must go with a mechanism that gives the control to the webmaster to put only the content they allow only where they allow.

Crawlability

Making crawlable an ajaxised website is one of the hardest challenges for a webmaster.
The feature must propose a declarative, simple and stupid way to ajaxise the website so the crawlers can browse it.

Bookmarkability and shareability

Often bound with the problem of making a website crawlable, bookmarkability is another challenge for the webmaster.
So as it browses the website, the user-agent must update the URL of the location bar and suggest to the visitor to bookmark (or share the URL) at any moment. And obviously, when bookmarked, a link must lead to the same content (except when the webmaster has updated the page).

The At Inclusion String

At inclusion is a part of the structure of URL (like the query string), which gives the user-agent the contents to be included in the document. An At Inclusion string is described by comma-separated series of pairs. Each pair contains :

• the key which is the ID of the element where to inject the contents. We also name it the target ID and we call the matching element the target element;
• the value which is the URL that leads to the contents to load. These contents are HTML content fragments.

I do not know which elements can be a target. See Which elements can be a target.

The At Inclusion string is located after the pathname, after the query string if it exists, and before the hash string. An example of At Inclusion string is :

@id1=/to/my/remote/content.html,id2=relativepath/to/content.html

and an example of a full URL having an At Inclusion string :

http://www.example.com/rep/mypage.html?query_key1=value1&@id1=/to/my/remote/content.html,id2=relativepath/to/content.html#hashstring

When you set the At Inclusion string, only the contents of the targets whose URL have changed are updated (and the document is not fully reloaded).

When you change the content of a target element, the old content is put aside and removed from the document by the user-agent. This old content will be referred below as the default content.

Dealing with nested At Inclusion pairs

Nested At Inclusion is the situation when you have a At Inclusion pair whose target exists only after another pair is loaded.

Let us name the first pair the dependent pair and the second the depended pair.
The URL with these two pairs must have this form :

http://mysite/document.html@...DEPENDED_TARGET=DEPENDED_URL,...,DEPENDED_TARGET.DEPENDANT_TARGET=DEPENDANT_URL

You would have noticed these two particularities :

1. the depended pair is specified before the dependent pair;
2. the dependent target is preceded by "DEPENDED_TARGET.". The benefit of such a method is that the user-agent knows the order of the HTML injection (first the depended target, then the dependent target);

If the dependent pair is the depended target of another pair, the target ID of the latter has to be preceded by its direct depended pair only:

http://mysite/document.html@...T1=URL1,...,T1.T2=URL2,...,T2.T3=URL3

Considering that example :

Main document:

<html> <head>...</head> <body> <div id='target1'></div> <div id='target2'></div> </body> </html>

content1.html :

<p>Content 1</p> <div id='target3'></div>

content2.html :

<p>Content 2</p>

content3.html :

<p>Content 3</p>

And we want to produce that result :

<html> <head>...</head> <body> <div id='target1'> <p>Content 1</p> <div id='target3'> <p>Content 3</p> </div> </div> <div id='target2'><p>Content 2</p></div> </body> </html>

We can see that target3 depends on target1. Then the following URLs are correct :

But these URL are incorrect :

At Inclusion operations

The At Inclusion operations consist in adding, replacing or removing pairs in the At Inclusion string. The string containing these operations will be referred further as the operation string.

To remove an existing pair and restore the default content in the target, the operation string must include the minus (-) operator, followed by the target ID. For example: '@-target1'.

To add a new pair or to replace an existing one, the operation string must include the plus (+) operator followed by the pair to add/replace. For example: '@+target1=url1'.
In the case of an addition, the pair is appended to the At Inclusion string, and in the case of a replacement, the new pair takes the place of the old one

Processing the requests

Security with HTTP headers

To avoid security issues, when processing the requests of the URLs in At Inclusion pairs, the user-agent sends the referer HTTP header (as usual), plus this header :

At-Inclusion-Target: TARGET_ID

Then, the server-side configuration tells if the targeted file is designed to be used as an inclusion. If this is the case, the server looks at the permissions to include the content of the file in the main document (using the referer) and at the specified target element (using At-Inclusion-Target). If the permission is explicitly given, the server sends back the content and includes in the response this header :

At-Inclusion-Allowed: true

That header means that the permission is expressly given to include that content. If the response does not include this header, the user-agent must consider the response as if it was Forbidden (403).
Thus, it avoids security issues if the server is not configured.

NOTE: a way to use CORS with At Inclusion needs to be discussed.

Returned status codes

If the status code is 200 or 304, the request has succeeded. Otherwise, if the status code is 301, 302, 303, or 307, see http://www.w3.org/TR/XMLHttpRequest/#infrastructure-for-the-send-method (for the same-origin case).

Other status code are treated as request errors.

Error handling

The request failed

Proposition 1 : If the request failed, the user-agent must show the error (HTTP status code + message) in the target element.

Proposition 2 : If the request failed, the user-agent must unload the whole document and display what the server returned instead (ex.: the 404 page).

The target does not exist

If the target does not exist, the user-agent must not request the URL, and must instead remove the pair from the At Inclusion string in the URL.
Then, it sends an error (or a warning?) to be displayed in the consoles, with that message : "the target was not found for the pair PAIR".

Pairs treatment

Here is an activity diagram of how the requests would be handled:

Setting At Inclusion pairs

Through HTMLAnchor Element

To create an At Inclusion hyperlink, the At Inclusion string must be specified in an href attribute of an anchor:

<a href='@target1=/to/content1.html,target2=/to/content2.html...'>Link text</a>

And one can also specify an operation string in it :

<a href='@+target1=/to/content1.html...'>Link text</a>

Through window.location API (JavaScript)

The location.atinclusion property gets or sets the At Inclusion string in the URL. Moreover, an operation string can also be given to the setter :

location.atinclusion = '@target1=/to/content1.html';
location.atinclusion; // == '@target1=/to/content1.html
location.atinclusion += ',target2=/to/content2.html';
location.atinclusion; // == '@target1=/to/content1.html,target2=/to/content2.html'
location.atinclusion = '@-target2,+target3=/to/content3.html';
location.atinclusion; // == '@target1=/to/content1.html,target3=/to/content3.html'


When an At Inclusion string or an At Inclusion operation string is given to location directly, the behaviour is the same as above :

location = '@target1=/to/content1.html';
location.atinclusion; // == '@target1=/to/content1.html'
...

Example of implementation in JS

A concrete prototype of implementation (in JS, though, but that is for an overview of the idea) is given here :

http://fflorent.github.com/HUX

In this site, each link uses At Inclusion. To test it, click on links in the menu. You will see the update of the URL in the location bar, for example :

http://fflorent.github.com/HUX/?@container=contents/features.html,feature=contents/features/atinclusion.html (NOTE: the content must be outdated)

This site uses At Inclusion through the HUX Framework, that I develop.
Some points have not implemented yet, though:

• the HTTP headers for security;
• the dependent pairs are not prefixed.

So, Framework of this kind could be used for retro-compatibilities (with a lower performance, though).

if you use IE 7/8/9, you may have noticed that At Inclusion pairs are converted to Hash bangs.

Remaining questions

Here are questions, needing to go further with that proposition

Using At

The At symbol is already used in URLs to identify a user (example : http://user:pwd@example.org/). This symbol seemed perfect to me, meaning put something at somewhere, but if that means that we cannot use this symbol anymore, another symbol must be found to replace it.

Extension for files with HTML fragments

Since we use files that do not have valid HTML content, is it pertinent to use the .html extension for these files? Perhaps the extension .htmlfr, that would stand for HTML fragment, would be a better choice ?

Deal with several elements having the same ID

Even if that it should never occur, how should the user-agent deal when the target ID of an At Inclusion pair matches with several elements ?

I think the question is easy to answer: the user-agent injects the content in the first pair in the DOM matching the ID, and then raises a warning.

Which elements can be a target ?

I would be very "liberal" with that question : any html element, descendant of <body>, that can have an ID attribute and at least a child element or text could be a target, except <script> and <style>

But what about elements that are not specified in the HTML5 specifications ?

Events

I think that there should be events that would go with the At Inclusion requests :

• on request begins
• on request succeeded
• on request error
• on targets updated (after request is succeeded)

CSS / Animations ?

I would also see the ability to apply specific CSS while the requests are running, and make animations before and after the targets are being updated.

The question is : how to trigger these animations ? Via Javascript ? Via CSS modifiers ?

Changing document title

When the "main" content of the page is changed, the document title is expected to change too.

However, I have no idea yet how to set it up exactly. Should we use special tags for HTML fragment files?

And also, due to the asynchronous property of At Inclusion, there should be a way to say which pairs can change the title.

My opinion is that one target, and one target of its child (i.e. dependent target), and one target of the latter's child, etc. can change it. So we are sure that if we have @a=u1,a.b=u2,b.c=u3,b.d=u4,c.e=u5,x=u5 , where a,b,c can change the title, whatever the load order is for these targets, c's title overrides b's title which overrides a's title. So c's title is always shown with this At Inclusion string.

Script execution order

Due to the asynchronous property of At Inclusion, the execution order of the <script> tags in HTML fragments must be thinked.

There are 2 scenarios for that :

• the user browses the website;
• the user loads a page using an URL with At Inclusion pairs.