Dropping the F-Bomb on Web Standards

photo credit: wiccked via photopin cc

In 2012, Mirriam-Webster’s dictionary added a definition for the F-Bomb.  Why?  Because the elite Mirriam-Webster work
d-wonk committee decided it was necessary to mint a “steamy new word”?  No, rather, because it is a well-established part of the common vernacular of the English language. There are occurrences of it going all the way back to 1988.  Not every slang term someone makes up will get into the dictionary. The Oxford English Dictionary has a vault full of millions of words that currently do not make the cut.  Many never catch on.  Some die out quickly and others change shape as they spread, that is what etymology is all about.  Some stagnate and maintain distinct and valuable regional meanings, and that is fine, but they aren’t part of the standard language.  The ones that are ultimately widely understood and eventually become commonly used are accepted and codified into the dictionary. Some words even become extinct.  In other words, the process of word standardization is evolutionary.

What’s this got to do with Web Standards?

There are actually many similarities between a dictionary which codifies and specifies the English language and a Web standard, but today the two work in nearly opposite ways*.

I didn’t come up with the dictionary metaphor (I dont know who did first).  I first heard Alex Russell mention it in 2011 in a Fronteers talk while I was trying my best to find a way to describe it succinctly myself and it really struck a chord with me.  It very simply illustrates in a way that is easily understood, not just the problem itself, but some excellent/proven solutions that we can use to solve them:  What we really need is a way to develop the slang of the Web and, as it catches on,  potentially mutates or dies, eventually have a way to recognize that, pick it up and codify it into the standards dictionary.

* Some attempts at seeing what people are doing outside of Web standards has happened, but for the most part, the real work of creation happens in a committee and the   dominance or extinction happens by browser vendors (who also dominate the committees).  This has, so far, left us with APIs that are often less than what we want, general slowness in rate of change and lots of other undesirable qualities for developers.

A Path for Natural Platform Evolution

photo credit: Kaptain Kobold via photopin cc

Yehuda Katz and Alex Russell gave an excellent presentation on the importance of Layering at the first W3C Technical Architecture Group F2F recently.  As you can see from the minutes, they returned on to this idea again with some positive/open responses from other member including questions and observations that further illustrate the language-link from Sir Tim Berners-Lee who seemed to express a lot of interest in parts.

It is critical to have competition/mutations and a population to evolve a platform – it is important to the long term health of the Web that we be able to evolve the slang of the Web (not just specify and release).  Yehuda called this a “Path for Natural Platform Evolution”.

What’s in it for me?  What do I have to do?

If you have ever participated in the open standards lists you know:  Most people aren’t that committed to bettering something that might make their lives easier years from now.   They have a job to do.   Within a month or two of following any list, you will begin to recognize the same small group of people who really contribute at that level.  It’s not that the general public is unwilling to contribute, it’s just really difficult to do and they don’t actually get anything out of it now.  This is distinctly different from words in a language – it’s very easy (and cheap) to pick up words that you find useful or descriptive and use them in your everyday parlance, on Facebook or Twitter.  You get something out of it when you find something becomes much easier to describe or maybe even makes you sound a little smarter or hipper than you might be without them.  Web standards on the other hand are kind of the opposite of that at the moment.

Cheap ways to help

1. Collect the slang:  One very cheap way that you can help if you have chrome is to install Meaningless and help collect anonymous statistics about the elements and approaches used in sites that you visit – this helps inform what slang is actually picking up and which is mostly just theory or fad.  What benefit do you, personally, get out of it right now?  Actually none, but it’s so easy, why not help
2. Provide the environment, use the slang:  Prollyfills (which I write about a lot) are to standards as slang is to words in the dictionary – they are essentially proposals that we hope catch on.  Using them right now actually does deliver value and it can be pretty cheap to try them out and provide some feedback.  If you’re very interested, join the Extensible Web Community group (see prollyfills link above) and help out – it’s open and you can do so in any way you like, and at your own pace.  You can comment, show us examples/use cases, contribute new ideas, or help create tests – or just help promote some proposal you really like.

Missing links….

Unfortunately this will only take us so far with the Web platform as it is today.  As Yehuda and Alex explained in their TAG presentation – it is hard to impossible to prollyfill some things in order for us to develop this slang in the real world: The platform doesn’t contain the right layers for people to step in and tweak/develop just one piece.  The only ones capable of doing this are browser vendors.  The architecture is lacking.

Thus, today, developers write increasing complex JavaScript: Re-implementing in order to emulate things that are already natively implemented in the browser just to make a single piece work a little better.

In order to fix this problem, we need to fix the gaps in the platform.  Some of this will take a while.  Luckily we have some good people working on it.  We’ve elected some good guys to the W3C Technical Architecture Group, both the membership and private encouragement we have gotten in the W3C Extensible Web Community Group have been encouraging  and there are spec authors like Tab Atkins who are helping to open new doors to make things like this more possible in places where they are currently very hard.

Related articles

• Bedrock (infrequently.org)
• Hoisted from the Comments (infrequently.org)
• How the Web Should Work (smus.com)

Off With Their Heads: Disband the W3C?

Just a few days ago, Matthew Butterick presented a talk entited “The Bomb in the Garden” at TYPO in San Francisco (poor title timing given recent events in Boston). In it, he concludes “the misery exists because of the W3C—the World Wide Web Consortium… So, respectfully, but quite seriously, I suggest: let’s Disband the W3C“. Ultimately he suggests that “...the alternative is a web that’s organized entirely as a set of open-source software projects.“

Butterick’s Points:

• It takes a really long time for standards to reach Recommendation Status (“the Web is 20 years old)
• The W3C doesn’t enforce standards
• Browser vendors eventually implement the same standards differently
• We fill pages with hacks and black magic to make it work
• Ultimately, what we wind up with still isn’t nearly good enough
• There is no good revenue model
• Newspaper and magazine sites all look roughly the same and are somewhat ‘low design’.

His presentation is definitely interesting and worth a read/view. In general, if you have been working on the Web a long time, you will probably experience at least some moments where you can completely relate to what he is saying.

Still, it seems a little Red Queen/over-the-top to me so I hope you’ll humor a little Alice in Wonderland themed commentary…

Why is a Raven Like a Writing Desk?

Michael Smith (@sideshowbarker to some) replied with some thoughts on it on the W3C Blog with a post entiteled “Getting agreements is hard (some thoughts on Matthew Butterick’s “The Bomb in the Garden” talk at TYPO San Francisco)” in which he points out in short, bullet-list form, several problems with Butterick’s statements about how W3C is misportrayed. The post is short enough and already bulleted so I won’t summarize here, instead I encourage you to go have a read yourself.  He closes up with the point that “Nowhere in Matthew Butterick’s talk is there a real proposal for how we could get agreements any quicker or easier or less painfully than we do now by following the current standards-development process.” (emphasis mine).

Indeed, the open source projects mentioned by Butterick are about as much like standards as a raven is like a writing desk and, in my opinion, to replace a standards body with a vague “bunch of open source projects” would send us down a nasty rabbit hole (or through the looking glass) into a confusing and disorienting world: Curiouser and curiouser.

“Would you tell me, please, which way I ought to go from here?”
“That depends a good deal on where you want to get to.”
“I don’t much care where –”
“Then it doesn’t matter which way you go.”
― Lewis Carroll, Alice in Wonderland

Still, I don’t think Butterick really means it quite so literally.  After all,  he holds up PDF as an ISO standard that “just works” and ISO is anything but an open source project like Wordpres.  In fact, PDF and ISO could have some of the same challenges laid against them.  For example, from the ISO website:

Are ISO standards mandatory?

ISO standards are voluntary. ISO is a non-governmental organization and it has no power to enforce the implementation of the standards it develops.

It seems to me that ISO and W3C have a whole lot more in common than they differ IMO:  Standards are proposed by stakeholders, they go before technical committees, they have mailing lists and working groups, they have to reach consensus, etc.  Most of this is stated in Michael’s post.  Additionally though, all PDF readers are not alike either: Different readers have different level of support for reflow and there is a separate thing called “PDF/A” which extends the standard (they aren’t the only ones) and adds DRM (make it expensive?).  Some readers/authors can accept links to places outside the file, some can’t.  Some can contain comments added by reviewers or markings, others can’t.   Etc.

You used to be much more…”muchier.”

I think that instead, Butterick is simply (over) expressing his frustration and loss of hope in the W3C:  ”They’ve lost their “muchness”.  You know what?  It really does suck that we have experienced all of this pain, and to be honest, Butterick’s technical examples aren’t even scratching the surface.  After 20 years, you really really think we’d be a little further along.

“I can’t go back to yesterday because I was a different person then.”
― Lewis Carroll, Alice in Wonderland

A lot of the pain we’ve experienced has taken place due to really big detour in the history of Internet standards: The ones we really use and care about were basically sort of put on hold and efforts mostly put toward a “something else”.  Precisely which something else would have made the Web super awesome is a little fuzzy, but whatever it was you could bet that it would have contained at least one of the letters “x” “m” or “l” and have contained lots of “<” and “>”‘s.  The browser maker with the largest market share disbanded their team and another major one split up.  It got so contentious at one point that the WHATWG was established to carry on the specs that the W3C were abandoning.

Re-muchifying…

While we can’t go back and fix that now, the question is:  Can we prevent the problems from happening again and work together to make the Web a better place?  I think we can.

“Why, sometimes I’ve believed as many as six impossible things before breakfast.”
― Lewis Carroll, Alice in Wonderland

The W3C is an established standards body with a great infrastructure and all of the members you’d really need to make something happen.  Mozilla CTO Brendan Eich had some good advice in 2004:

What matters to web content authors is user agent market share. The way to crack that nut is not to encourage a few government and big company “easy marks” to go off on a new de-jure standards bender. That will only add to the mix of formats hiding behind firewalls and threatening to leak onto the Internet.

Luckily, it seems that the W3C has learned some important lessons recently.  More has happened to drive Web standards and browser development/interoperability forward in the past 2-3 years than happened in the 6-7 years combined and more is queued up than I can even wrap my head around.  We have lots of new powers in HTML and lots of new APIs in the DOM and CSS.  We have efforts like Test the Web Forward uncovering problems with interoperability and nearly all browsers becoming evergreen – pushing out improvements and fixes all the time.  We also managed to get some great reformers elected to the W3C Technical Architecture Group recently who are presenting some great ideas and partnership and cooperation between W3C and other standards bodies like ECMA/TC-39 (also making excellent progress) are beginning.   I believe that we can all win with community participation and evolution through ideas like prollyfill.org which is trying to team up the community with standards groups and implementers to create a more nimble and natural process based on evolutionary and open ideas… Perhaps that might sound like a marriage of open source ideas and standards that Matthew Butterick would be more happy with… Maybe I should send him an email.

So what do you think?

“Do you think I’ve gone round the bend?”
“I’m afraid so. You’re mad, bonkers, completely off your head. But I’ll tell you a secret. All the best people are.”
― Lewis Carroll, Alice in Wonderland

Logical Psuedo Selectors: A Proposal

When you get right down to it, CSS rules select matching sets of elements.  Sets and logic gates are two of the most fundamental and powerful concepts in computer science – at some level it is upon their backs that just about everything else builds.  Despite this, until pretty recently CSS has never had features that hinted at the concept of a set and that seems a shame because integrating the language of logic/sets could be a powerful combination.

What’s changing?

After many years during “the great stagnation” HTML and CSS are moving along quickly again. HTML has increased our ability to express semantics and CSS is adding new selector powers.  Selectors Level 3  is a done deal, work is well underway on Selectors Level 4 and we’ve already got a wiki of features for Selectors Level 5.  Likewise, we are adding long-needed features like Regions, scoped stylesheets, Shadow DOM and Generated Content.  All of these things combine to create a really positive future where CSS can really begin to live up to its potential and the visuals pertaining to good structures can managed via CSS without requiring changes to markup.

Two pseudo-classes in particular – in a group called Logical Combinators - currently :matches (some implemenations call it :any) and :not begin to bring with them some interesting possibilities.  Currently they are very limited and can only accept simple (or compound – depending on the level) selectors, but eventually they could accept complex selectors.  When that day comes we could begin talking about things in terms of sets.

A Proposal

4 Logical Combinators which can take complex selectors:

• :anyof (:matches?) Filters elements based on whether they match ANY of the provided selector(s) (that is, selectors are OR’ed together) which may express different paths in the same tree.
• :allof
  Filters elements based on whether they match ALL of the provided selector(s) (that is, selectors are AND’ed together) which may express different paths in the same tree.
• :noneof (:not ?) Filter elements based on whether they match NONE of the provided selector(s) (that is, selectors are NOR’ed together) which may express different paths in the same tree.
• :oneof
  Filters elements based on whether they match EXACTLY ONE of the provided selector(s) (that is, selectors are XOR’ed together) which may express different paths in the same tree.

An Example:
Given some rich markup that describes a structure in some local semantic terms (semantics important to my domain)…

I can use logical sets to add styles…

And they would style up as…

Prollyfilling…

All of the above are prollyfilled currently and come “out of the box” with hitchjs (works in IE9 and all evergreen browsers) – they are all prefixed with -hitch-*.  If you’d like to play around with it, simply add the following script include:

<script type="text/javascript" src="http://www.hitchjs.com/dist/hitch-0.6.1.min.js"></script>

and add a data attribute to any <style> or <link> tag which contains hitch prollyfilled rules, for example:

Read the original docs we wrote for this.

Regressive Disenfrancishement: Enhance, Fallback or Something else.

My previous This is Hurting Us All: It’s time to stop…” seems to have caused some debate because in it I mentioned delivering users of old/unsupported browsers a 403 page.  This is unfortunate as the 403 suggestion was not the thrust of the article, but a minor comment at the end.  This post takes a look at the history of how we’ve tried dealing with this problem, successes and failures alike, and offers some ideas on how an evergreen future might impact the problem space and solutions going forward.

A History of Evolving Ideas

Religious debates are almost always wrong: Almost no approach to things is entirely meritless and the more ideas that we mix and the more things change the more we make things progressively better.  Let’s take a look back at the history of the problem.

In THE Beginning….

In the early(ish) days of the Web there was some chaos: vendors were adding features quickly, often before they were even proposed as a standard. The things you could do with a Web page in any given browser varied wildly. Computers were also more expensive and bandwidth considerably lower, so it wasn’t uncommon to have a significant number of users without those capabilities, even if they had the right “brand”.

As a Web developer (or a company hiring one), you had essentially two choices:

• Create a website that worked everywhere, but was dull an non compelling, and used techniques and approaches which the community had already agreed were outdated and problematic – essentially hurting the marketability and creating tech debt.
• Choose to develop better code with more features and whiz/bang – Write for the future now and wait for the internet to catch up, maybe even help encourage it and not worry about all of the complexity and hassle.

“THIS SITE BEST VIEWED WITH NETSCAPE NAVIGATOR 4.7 at 800×600″ 

Many people opted for the later choice and, while we balk at it, it wasn’t exactly a stupid business decision.  Getting a website wasn’t a cheap proposition and it was a wholly new business expense, lots of businesses didn’t even have internal networks or significant business software.  How could they justify paying people good money for code that was intended to be replaced as soon as possible?

Very quickly, however, people realized that even though they put a notice with an “Get a better browser” kind of link, that link was delivered along with a really awful page which makes your company look bad.

Browser Detection

To deal with this problem sites started detecting your browser via user-agent and giving you some simpler version of the “Your browser sucks” page which at least didn’t make them look unprofessional: A broken page is the worst thing your company can put in front of users… Some people might even associate their need for a “modern browser” as “ahead of the curve”.

LIAR!:  Vendors game the system

Netscape (at this point) was the de-facto standard of the Web and Microsoft was trying desperately to break into the market – but lots of sites were just telling IE users “no”.  The solution was simple:  Lie.  And so it was that Microsoft got a fake ID and walked right past the bouncer, by publicly answering the question “Who’s asking?” with “Netscape!”.

Instead of really fixing that system, we simply decided that it was too easy to game and moved on with other ideas like checking for Microsoft specific APIs like document.all to differentiate on the client.

Falling Back

As HTML began to grow and pages became increasingly interactive, we introduced the idea of fallback. If a user agent didn’t support script, or object/embed or something, give them some content. In user interface and SEO terms, that is a pretty smart business decision.

One problem: Very often, fallback content wasn’t used. When it was, the fallback usually said essentially “You browser sucks, so you don’t get to see this, you should upgrade”.

the CROSS browser era and the great stagnation

Ok, so we have to deal with more than one browser and at some point they both have competing ideas which aren’t standard, but are far too useful to ignore.  We create a whole host of solutions:

We came up with safe subsets of supported CSS and learned all of the quirks of the browsers and doctypes, we developed libraries to create new APIs that could switch code paths in and do the right thing with script APIs.

As you would expect, we learned things along the way that seem obvious in retrospect: Certain kinds of assumptions are just wrong.  For example:

• Unexpected vendor actions that might increase the number of sites a user can view with a given browser isn’t unique to Microsoft. Lots of solutions that switched code paths based on document.all started breaking as Opera copied it, but not all of Microsoft’s apis.  Feature detection is better than basing logic on assumptions about the current state of vendor APIs.
• All “support” is not the same – feature detection alone can be wrong.  Sometimes a standard API or feature is there, but it is so woefully incomplete or wrong that you really shouldn’t use it.

And all of them still involved some sense of developing for a big market share rather than “everyone”.  You were almost always developing for the latest browser or two for the same reasons listed above – only the justification was even greater as there were more APIs and more browser versions.  The target market share was increasing, but not aimed at everyone – that would be too expensive.

Progressive Enhancement

Then, in 2003 a presentation at SXSW entitled “Inclusive Web Design For the Future” introduced the idea of “progressive enhancement” and the world changed, right?

We’re all familiar with examples of a list of links that use some unobtrusive JavaScript to add a more pleasant experience for people with JavaScript enabled browsers.  We’re all familiar with examples that take that a step further and do some feature testing to take this a bit further and make the experience still a little better if your browser has additional features, but still deliver the crux content.  It gets better progressively along with capabilities.

Hold that Thought…

Let’s skip ahead a few years and think about what happened:  Use of libraries like jQuery exploded and so did interactivity on the Web, new browsers became more mainstream and we started getting some forward progress and competition again.

In 2009, Remy Sharp introduced the idea of polyfills – code that that fill the cracks and provides slightly older browsers with the same standard capabilities as the newer ones.  I’d like to cite his Google Plus post on the history: 

I knew what I was after wasn’t progressive enhancement because the baseline that I was working to required JavaScript and the latest technology. So that existing term didn’t work for me.

I also knew that it wasn’t graceful degradation, because without the native functionality and without JavaScript (assuming your polyfill uses JavaScript), it wouldn’t work at all.

In the past few years, all of these factors have increased, not decreased.  We have more browsers, more common devices with variant needs, more OS variance, and an explosion of new features and UX expectations.

Let’s get to the point already…

The presentation at SXSW aimed to “leave no one behind” by starting from literally text only and progressively enhancing from there.   It was in direct opposition to the previous mentality of “graceful degradation” – fallback to a known quantity if the minimum requirements are not met.  

What we’re definitely not generally doing, however, is actually living up to the full principles laid out that presentation for anything more than the most trivial kinds of websites.

Literally every site I have ever known has “established a baseline” of what browsers they will “support” based on market-share.  Once a browser drops below some arbitrary percentage, they stop testing/considering those browsers to some extent.  Here’s the thing:  This is not what that original presentation was about.  You can pick and choose your metrics, but the net result is that people will hit your site or app with browsers you no longer support and what will they get?

IE<7 is “dead”.  Quite a large number of sites/apps/libraries have announced that they no longer support IE7, and many are beginning to drop support for IE8.  When we add in all of the users that we are no longer testing for and it’s becoming an a significant number of people… So what happens to those users?

In an ideal, progressively enhanced world they would get some meaningful content, progressive graded according to their abilities, Right?

But in Reality…

What does the online world of today look like to someone, for example, still using IE5?

Here’s Twitter:

Twitter is entirely unusable…

And Reddit:

Reddit is unusable… 

Facebook is all over the map.  Most of the public pages that I could get to (couldn’t login) had too much DOM/required too much scroll to get a good screenshot of – but it was also unusable.

Amazon was at least partially navigable, but I think that is partially luck because a whole lot of it was just an incoherent jumble:

Oh the irony.

I’m not cherry picking either – most sites (even ones you’d think would because they aren’t very feature rich or ‘single page app’ like) just don’t work at all.  Ironically, even some that are about design and progressive enhancement just cause that browser to crash.

FAIL?

Unless your answer to the question is “which browsers can I use on your site and still have a meaningful experience?” is “all of them” then you have failed in the original goals of progressive enhancement.  

Here’s something interesting to note:  A lot of people mention that Yahoo was quick to pick up on the better ideas about progressive enhancement and introduced “graded browser support” in YUI.   In it, it states

Tim Berners-Lee, inventor of the World Wide Web and director of the W3C, has said it best:

“Anyone who slaps a ‘this page is best viewed with Browser X’ label on a Web page appears to be yearning for the bad old days, before the Web, when you had very little chance of reading a document written on another computer, another word processor, or another network.”

However, if you read it you will note that it identifies:

C-grade browsers should be identified on a blacklist.

and if you visit Yahoo.com today with Internet Explorer 5.2 on the Mac here is what you will see:

Your browser sucks.

Likewise, here’s what happens on Google Plus:

You must be at least this tall to ride this ride…

In Summary…

So what am I saying exactly?  A few things:

• We do have to recognize that there are business realities and cost to supporting browsers to any degree.  Real “progressive enhancement” could be extremely costly in cases with very rich UI, and sometimes it might not make economic sense.  In some cases, the experience is the product.  To be honest, I’ve never really seen it done completely myself, but that’s not to say it doesn’t exist.
• We are right on the cusp of an evergreen world which is a game changer.  In an evergreen world, we can use ideas like pollyfills, prollyfills and “high end progressive enhancement” very efficiently as there are no more “far behind laggards” entering the system.
• There are still laggards in the system and there likely will be for some time to come – we should do what we can to get as many of them who can update to do so and decrease the scope of this problem.
• We are still faced with choices that are unpleasant from a business perspective for how to deal with those laggards in terms of new code we write.  There is no magic “right” answer.
• It’s not entirely wrong to prevent yourself from showing your users totally broken stuff that you’d prefer they not experience and associate with you.  It is considerably friendlier to them if you literally write them off (as the examples above do) anyway and there is at least a chance that you can get them to upgrade.
• In most cases, however, the Web is about access to content – so writing anyone off might not be the best approach.  Instead it might be worth investigating a new approach, here’s one suggestion that might work for even complex sites:  Design a single, universal fallback content (hopefully one which still unobtrusively notifies the user why they are getting it and prompts them to go evergreen) which should work on even very old browsers to deliver them meaningful, but probably comparatively non compelling content/interactions and deliver that to non-evergreen browsers and search engines.  Draw the line at evergreen and enhance/fill from there.

This is Hurting Us All: It’s time to stop…

The status quo with regard to how users receive browser upgrades has changed dramatically in the last few years – modern browsers are now evergreen, they are never significantly out of date with regard to standards support (generally a matter of weeks), and ones that are not (generally mobile) are statistically never more than a year or so back.  This means that in 2013 we can write software using standards and APIs created in, say, 2012 instead of 2000 – but by and large we don’t – and that is hurting us all.

It’s not about a specific version…

Historically, a large number of users of got their browser upgraded only as a side-effect of purchasing a new computer.  Internet Explorer 6 was released toward the end of 2001 – well over a decade ago – pre-9/11/2001.  It wasn’t fully compliant with the standards of the day – and for five years, nothing much happened in IE land, so anyone who bought a new machine until about 2007 was still getting IE6 for the first time.  This meant that until very recently many businesses and Web authors still supported IE6 – and it was only with active campaigning that we were finally able to put a nail in that coffin.

So, now we’re all good – right?  Wrong.

A great number of companies have followed their traditional policies and took this to mean that they must now offer support back to IE7, some have even jumped to IE8 which seems like a quantum leap in version progress, but…  If you write Web sites or run a company that does, this is now hurting us all: it’s time to stop.  

The problem is in the gaps not the versions…

Let me add some data and visuals to put this into perspective…   If you were to create a giant table of features/abilities in the Web Platform today in most evergreen and the vast majority of mobile browsers and compare them with the abilities of non-evergreen IE browsers, remove the features that have been in there for more than a decade and then zoom WAY out, you would see something like the image here (and then you’d have a lot of scrolling to do).  Rows represent features, the green columns on the left represent evergreen browsers and the red columns on the right represent IE10, 9, 8, 7, 6 (in that order).

If you want the actual dirty details and to see just how long my (incomplete) list of features really is, you can take a look at:

•  this table which uses data from caniuse.com and shows that 109 out of 134 features that they track are supported in most evergreen browsers, but pretty much none of them before IE9
•  this table which uses data from webbrowsercompatability.com and shows 460 of a total 1403 properties/features that they are testing for are available in most evergreen browsers but, again, pretty much none of them before IE9.  

Now, if you really look at the data, both are slightly flawed, but the vast majority of the picture is crystal clear…

• There are an ever increasing number of gaps in ability between evergreen browsers and old IE.  More importantly is where the gaps are: Many are some of the most powerful and important sorts of features to come to the Web in a really long time.
•  We can’t wait years for these features because meanwhile the user/business expectations for features made possible by these APIs keep increasing.
• Solutions/workarounds for missing features are non-standard.  They start getting bulky as you increase their number, they are less maintainable, require more code and testing (anecdotally, I have worked on a few small projects that spent almost as much time working out cross browser problems as the entire rest of the project).

How it hurts us all…

Directly, this is hurting you or your company because it means you spend a significant amount of time and money developing for and supporting browsers which are so far behind the curve – and the work you are doing is already out of date.

What’s more though, by supporting them you are effectively enabling these users to continue inadvertently bad behavior which is free to fix – and the longer they stay around, the more other companies follow suit.  You are creating a self-fulfilling prophecy:  My users won’t upgrade.

What can we do?

Luckily, this is actually a really solvable problem… All we have to do is get users to upgrade their silly browsers – and it’s free.

I propose that we set two dates as a community and do the following:

• Try to get cooperation from major sites like we did for SOPA and make sections of the interwebs go semi-dark for those users (like we did for SOPA) on the first date – while we tell them about the second date (next bullet)
• Upon the second date, we collectively stop serving them content and start serving them up 403′s (I’d propose July 4th - Independence Day in the US, but that doesn’t internationalize well).

I think there are a very small number of users who still don’t have high speed access, for those people, you could recommend them to Best-Buy where they can pick up a copy of Google Chrome on CD.  If you have other ideas, want to help coordinate a date, want to submit scripts that people can use to black out their site or even want to share another way that users can get disks by mail, leave a comment.

Clarification: I am not suggesting that the entire Web start serving 403 responses to anyone with an older version of IE here – I am suggesting that sites stop support/new development for those browsers.  Eventually, for some classes of sites, this will mean that their site is actually unusable (try using twitter or google plus with IE4, I haven’t, but I bet you won’t get far) – if that is the case, a 403 page asking users to upgrade is an entirely appropriate response which is actually better than a broken user experience.  This is about helping users to help us all. 

Our support of non-evergreen browsers is killing us all, let’s work together to spread the word and fix that pro-actively, as a community.

Thanks to Paul Irish and Cody Lindey for helping to hook me up with some data.

Dear W3C TAG

Last week following in tradition, TAG had an unofficial teleconference introducing old and new members and discussing the future and focus of TAG itself.  During this discussion (and prior to the election) the idea of bringing TAG closer to/making it more known in and relevant to the development community was raised (and seemingly supported by the co-chair).  Given this, I have offered the input of one developer (me) and expand on some of those points and their relevance here for anyone who cares to understand why I think these are important or relevant.

Read “build concensus” in the charter as “in the larger community”

Convince us.  Get us interested, excited and involved.  Given it’s very small size and makeup it provides (or can provide) a unique bully pulpit that has been under-utilized. One of the few things explicitly within TAG’s charter is to build consensus and I think that that has to read (though it seems to historically not been read as) as within the larger community.

Foster a More Direct Relationship With The Community

In order to use the bully pulpit, first people have to care.  Most developers didn’t even know that TAG existed until very recently.  The connections/credentials that the newly elected members have in the developer community is just one reason I actively campaigned on their behalf.  TAG members should hear, if not experience first hand, the pain and feedback that comes from the wider development community. We don’t ask that they solve each issue, merely that we feel that they understand them and aren’t off in an ivory tower somewhere saying “Let them eat cake” while we all starve.

A more public presence of TAG / members

One of the most interesting and effective things to happen in semantic web, in my opinion, was when TimBL did a couple of Ted Talks.  We need representatives that lots of people are willing to listen to, and I think we’ve just elected some good ones.  I for one would love to see blogs and talks by members (taking their TAG hats off if necessary) just to let people know that they are out there, involved and thinking deeply about significant issues whether those issues necessarily get picked up and addressed formally by TAG or not.

Back in the late 1990′s and early 2000′s TimBL wrote a series of thoughts on design issues (http://www.w3.org/DesignIssues) including one that I love called “The Evolution of a Specification” (http://www.w3.org/DesignIssues/Evolution.html) which contains a lot of great insights and thoughts.  With a decade and a half in retrospect, I’d love to hear updated thoughts from him and others.

Consider how W3C process/policy affect long term health

Noah (the TAG co-chair) said,  “The TAG’s charter is to focus on the long-term architectural viability of the Web, but we need to do a much better job of having impact year-by-year. My perception is that these goals are sometimes in tension. How do we help a community that’s implementing “early and often” to also build an architecture that’s clean and will scale for decades?”

Historically within W3C at at large it does seem that those two things are often (maybe always) tension, but it doesn’t seem that they inherently need be and I think that that is something I would love to see TAG discuss.  A healthy architecture requires a system/processes that promotes health and I feel that there is currently a lot that doesn’t.

An inordinate amount of fuss and time and energy is spent at what really amounts to bike-shedding and formalities that are taken because of some other obscure formality which – at the end of the day, to many of us it seems at best silly and at worst confusing.  I’d like to see TAG comment on whether fixing some of this is actually part of creating a stable and healthy architecture – I think it is.

I think that you can sum up a lot of my thoughts on this as: Paving cowpaths implies that cowpaths can be created in the first place and really, they often can’t, and I think that is unhealthy and leads to an inordinate amount of missteps.

Layers, layers, layers

If TAG deals with helping to create layers and fundamental principles that allow, maybe even encourage evolution, then it’s work (and W3C’s) will be relevant for decades to come, and we will be able to collect data to show it.  Even if certain layers are improved or replaced, we will no more mourn them than we do the loss of manual typesetting when we read a book.

What we can learn from phone numbers?

The co-chair also cited the phone number system created 90+ years ago as example of something that has stood the test of time and a kind of model of what TAG needs to do as one of its larger goals ”… part of the TAGs role is to help the community build a Web that will be viable in growing in 50 years, and maybe 100.”

Here are what lessons I think are worth drawing from that:

• Like much of the success of the early Web it’s success appears to be largely because it is really about a targeted and small thing that wasn’t necessarily technically the best – it was the best that they could sell.
• The inventors never imagined, or tried to imagine, all of the myriad of things that telephone numbers would be used for 50 and 100 years out – and if they had, you can bet they would have missed the mark substantially.
• While it was created in 1923, it wasn’t really the world-wide standard until it beat out all of the competition much later.  It is entirely plausible that in the intervening years something better could have come along and displaced it.
• It was decoupled from the larger system enough in design to allow that phones, transmission lines, and wholly new ideas using this infrastrucuture could all evolve more or less independently.

I am earnestly looking forward to hearing/reading more from this group and hope that other voices in the community (like you?) will contribute to the conversations and help make our collective future a better place.

Hours Remain: Help the Future of Web Standards

The interwebs need your help. If you’ve read my previous post The New Gang of Four or you follow Web standards work/W3C closely you probably know what I am talking about:  The W3C Technical Architecture Group elections end today.  Polls close at midnight US Eastern Standard Time tomorrow (1/9/2013) and it’s still close – both sides are encouraging voters and we are close to record turnout.  Chances are pretty good that you might not know someone from one of these 387 companies who get to vote, so what can you do to help?  Well, even if you don’t know someone – chances are pretty good that someone you know does – or someone they know.  We are remarkably interconnected and very few of them actually cast their votes and smaller companies wield an equal amount of power in this case.

So here is what I am asking from you – just pass it on:

• If you have google+ :  Paul Irish lays it out here https://plus.google.com/113127438179392830442/posts/ix4bWGNRy33
• If you tweet: Rick Waldron hates my title, but endorses here: https://twitter.com/rwaldron/status/277197567010426881 (#tag2013)
• If you have Facebook: http://www.facebook.com/brian.kardell.71/posts/132649506891893

I If you blog about it, let me know in the comments or via twitter @briankardell and I will (probably) RT.  If you have lots of followers:  Your endorsement could make the difference and I would love to add it to the list.

To some extent, this is something of a battle for the soul of the Web and our encouragement has already had positive effects on this group who is taking a hard look at things.

Fingers crossed.

The New Gang Of Four

For programmers, when we hear “Gang Of Four” we pretty much immediately conjure up an image of the blue and white book that most of us have sitting on a shelf somewhere in which four very smart people sat down and documented design patterns which would shape the way that people think about writing software for many years to come.  However, the concept a “Gang of N” is used frequently in politics the world round to denote a group of, at least somewhat like-minded people who form a voting block big enough to matter and thus wield a greater degree of power than any one of them could individually… Power which is often used to bring about significant changes rapidly.

The Powers that Be

Most developers have probably never heard of the W3C Technical Architecture Group, so if you haven’t, here it is in a nutshell:  This is a very small group of people at the W3C - a mere 9 people to be exact, chaired by Tim Berners-Lee himself – responsible for sort of envisioning the architecture of the Web itself and championing it.  I’m not sure exactly how much formal power they have, but in the very least it wields the power of the bully pulpit at the W3C and therefore to some significant extent over the actual focus that a lot of big groups drive toward.

The Need for Change

I will let you read more in the links below, but to sum up: Most of the stuff that this group has historically concerned itself with has little to do with the reality of some of the most important pieces of the Web platform that most of you reading this live and breathe every day.  If you want to think deep thoughts about how the Web is documents full of data interlinked through URIs – this is traditionally the group for you.  Don’t get me wrong – I’m not trying to trivialize it, but the Web is about so, so much more than this group has actively discussed and in my opinion this is creating a sad new reality where it seems that the importance and influence of the once proud W3C is less important to the everyday developer. If you now reference the WHAT-WG HTML Living Standard over the current W3C version, prefer JSON despite W3C’s enormous push of “technically superior” XML family of languages and tools and really want more focus on things that you actually deal with programmatically – then you see the problem.

One GIANT Chance to Make a Big Change

Only 5 of the 9 positions in that group are elected (Sir Tim is the chair and 3 positions are appointed – hmmm).  However, it just so happens that this time around 4 of those 5 spots are up for grabs… and somehow miraculously we managed to get 4 people officially nominated who could really shake things up!

• Anne Van Kesteren
• Alex Russell
• Yehuda Katz
• Marcos Caceres

If you don’t know who these guys are – I think you just aren’t paying a whole lot of attention…   They are smart, super active (dozens of influential working groups and important open source projects) and not the sort of guys who sit quietly and go along if they disagree.  A few of them have written some good articles about what they think is wrong with TAG now and what they plan to do if elected [1][2][3][4][5].

Basically - Make it concentrate more on the stuff you really care about:

• Layered Architecture – increasing levels of abstraction
• Close coordination with ECMA
• Extensibility – Web Components / Shadow DOM – new APIs including composition and templates
• Bring a level of “from the trenches” representation.
• In short:  Advocate and use those powers for what people like us really care most about.

Let’s Get These Guys Elected!

Here’s what you can’t do: Cast an actual vote.  Only the 383 member organizations of W3C actually get to vote – but they get to vote once for every open slot.

Here is what you can do:  Lobby…. Use the power of social media to promote this article saying HEY MEMBER ORGS — THESE FOUR ARE WHO WE WANT  YOU TO CAST YOUR VOTE FOR… Tweet… +1… Like… Blog… Make Memes…

C’mon interwebs – do what you do so well…

W3C Extensible Web Community Group

The Web requires stability and a high degree of client ubiquity, that’s why W3C is important. However, that’s also part of the reason that standards develop at a comparatively, (sometimes seemingly painfully) slow rate: They take time, deliberation, testing and mutual agreement between a number of experts and organizations with significantly different interests.

While we frequently complain, that is actually a Really Good Thing™. Standards take a while to “cook” and the Web would be a real mess if every idea became a native standard haphazardly and with great speed:  Native implementations have a way of being hard to kill (or change significantly) once the public has free access to them – the general mantra is “don’t break it” – and that makes iteration/significant evolution artificially harder than it needs to be.  It seems that the pressures and incentives are mixed up.

The W3C Extensible Web Community Group was founded with a vision toward supplementing the traditional model with a new one which we believe is a better path forward in most cases.

Polyfills and Prollyfills…

Polyfills are a well-known concept and it’s an excellent, visualizable name for it:  Something that “fills the holes” here and there on browsers which are behind the curve with regard to implementation of a pretty mature spec.  However, since the term was coined by Remy Sharp a few years ago, and it has become an increasingly popular practice, its meaning has become somewhat diluted.

No longer is it the case that they are merely “filling in a few holes” in implementations based on mature specs – more and more often it is more like they are building whole new annexes based on a napkin sketch.  Within a very short period of time from the first announcement of a draft, we have some “fills” that provide implementation.

Given this contrast, it seems we could use a name which differentiates between the two concepts.  For a while now a few of us have been using different terms trying to describe it.  I’ve written about the concept in the past and it is the subject of Borris Smus’ excellent article How the Web Should Work.  Until recently the best term we could come up with was “forward polyfill”.  Then, on October 12, 2012 Alex Sexton coined the term “prollyfill” on Twitter.

October 12, 2012 “@SlexAxton: Prollyfill: a polyfill for a not yet standardized API”  

The Benefits of Prollyfilling

One thing is clear about the idea of prollyfills:   If we get them “right” they could radically improve the traditional model for standards evolution because they have a few very important benefits by their very nature. Most of this benefit comes from the simple decoupling from a browser release itself.  Since lots more people can contribute to their creation and you only need one implementation, they can be developed with much greater speed and by a larger community.  Such an approach also puts the author of a site in control of what is and isn’t supported.  In the traditional model, an author has no ability to change the speed at which native features are implemented nor what browser users will use to view it with.  Using prolyfills could allow the author to rely on JavaScript to implement the features needed, with only degraded performance, so this is a huge advantage in general.  However, even more of an advantage is realized in that scenario is that it allows multiple competing APIs or even variant APIs to co-exist while we round the edges and iterate because the author can choose which one they are going to use – that is pretty much impossible with the traditional model.  Iteration in the traditional model is likely to cause breakage which is a deterrent to use and therefore severely limiting in terms of how many use cases will be considered or how many authors can give meaningful feedback.   Iteration in this model breaks nothing, APIs and drafts can compete for users, get lots of feedback and ultimately if there is a clear winner it is evident from actual data before native code ever has to be written.

The value of the ability to compete and iterate freely should not be under-estimated – it is key to successful evolution.  Take, for example, the “nth” family of selectors.  They took a long time to come to be and it would appear that most people aren’t especially happy with them – they aren’t often used and they are often mis-understood.  Given the ability to create prollyfills for selector pseudo-classes, it is unlikely that this is the means of accomplishing use cases that would have won out, yet those limited resources ultimately spent an awful lot of time on working out details, drafting prose, presenting, getting agreement, implementing, going through processes about prefixing, optimizing implementations, etc.  In fact, the concept of nth was being discussed at W3C least as early as 1999 and parts of what was discussed was punted to CSS Selectors Level 4.  There was a definite effort to “answer all of the nth questions now,” and while what we have might be an “academically really good answer”  - it’s hard to argue that it’s really caught on and been as useful as some other things.  It’s easy to speculate what might have been better but the truth is, we’ll never know.

The truth is, currently a very small number of people at W3C do an enormous amount of work.  As of the most recent TPAC, the W3C CSS Working Group alone currently has 58 “in process” drafts and only a handful of editors.   This means that a significant number of them are going to be de-prioritized for now in order to focus on the ones with the most support or that are further along and new ideas probably won’t be undertaken for a time.  While they are trying to streamline the process, it does tend to go in fits and starts like this… Without a doubt, several of those 58 will be years and years in the making.

If instead, these same individuals could participate in less demanding, more advisory fashion to a satellite group of Web developers submitting experimental works and after a certain threshold could take over a fairly robust and reasonably mature draft, it is easy to imagine that things could evolve faster and those resources could focus a lot more.  Of course, browser manufacturers and standards bodies could participate in the same manner:  Adobe recently followed this model for their Draft Proposal on CSS Regions, Mozilla is working on a x-tags which implements part of the still very early Web Components API via script and as a general practice, the ECMA team does this pretty often.  These are very positive developments.

The Challenges of *lyfilling.

Currently every fill is generally implemented as a “from the ground up” undertaking, despite the fact that there is potentially a lot of overlap in the sorts of things you need to implement them.  For example: If you are filling something in CSS, for example, you have to (at a minimum) parse CSS.  If you are filling a selector pseudo, you’ve got a lot of figuring out, plumbing and work to do to make that work and efficient enough.  If you are filling a property, it’s potentially got a lot of overlap with the selector bit.  Or perhaps you have a completely new proposal that is merely “based on” essentially a subset of some existing Web technology, like Tab Atkin’s Cascading Attribute Sheets - it’s pretty hard to start at square one and that means that often these fills are comparatively low fidelity.

Another challenge with polyfills and prollyfills is how to write them effectively (best practices), how to make them discoverable and communicate what needs to be communicated about the degree of parity that they provide or their method/degree of forward compatibility.

Charting the Uncharted Waters

It seems clear that we could do with some cooperation and potentially some robust and well thought out and tested prollyfills for native APIs that would make some of this easier.

There really is nothing like “caniuse” for polyfills detailing compatibility, parity with a draft or method/degree of forward compatibility.  Likewise, here is also no such thing for prolyfills – nor is there a “W3C” sort of organization where you can post your proposal, discuss, get people to look at/contribute to your prose/examples, ask questions, etc.  No group creating and maintaining test cases or helping to collect data and work with willing W3C members to help make these things real or prioritized in any way.

In short, there is no community participation acting as a group interested in this subject.  These are gaps we hope to fill (pun intended) with the W3C Extensible Web Community Group.  We’re just getting started, but we have created a github organization and registered prollyfill.org.  Participate or just follow along in the conversations on our public mailing list.

What Do You Call It?

A long time ago, we just had “libraries” on the Web. Then we wound up with “shims” and still later “polyfills”. If you have ever read Remy Sharp’s article “What is a polyfill?” then you might understand why I find myself searching for the right word for this new paradigm I have noticed developing and why none of these existing words seem adequate or appropriate: An increasing number of proposals for standards are coming with open source implementations that allow users to try them out, think out it, ask questions and help better the proposal itself. Because solutions are not provided by a browser, but are “opt in” by the author (like libraries, polyfills and shims) there is no technical problem created by people writing their sites using different versions of it and may of them provide a very similar but ever so slightly different api than the native one they are proposing, so there is no forward compatibilty risk either.

HitchJs work like this, as do adobe’s recent proposal and x-tags and others (the practical benefits of this model is discussed a bit in our recent post About :time).

So, if it’s not neatly definable as any of those things, that raises the question: What do you call them?

I have heard proposals like “polymimic” and “draft shimplementation”. I’d like to hear your proposals…

————-
Update: A few others have recently latched on to some similar ideas since this was written, the best term we have come up with thus far is “forward polyfill”. My friend and CSS Custom Properties alternative proposal co-author Francois has also recently added a blog post arguing that TypeScript is actually a pretty good example of exactly this sort of thing. It’s worth reading if you are having deep thoughts about this sort of thing: read on.