HTML5 discussion: Interface

[petdance]

Before any code gets written, I'd like to hash out the interface for HTML::Tagset 5.00+. Following is my proposal and I welcome your feedback. Some points may run counter to prior things I've said. This is what I think makes most sense right now, and after seeing the discussion we've been having the past few weeks.

Guiding principles

• Upgrading to HTML::Tagset 5.00 will not break existing code.
• use HTML::Tagset; must give you HTML4 data, in existing hash format.
• If you want to have HTML5 support, you must:
  • Install HTML::Tagset 5.00 or higher.
  • Change your use statement: use HTML::Tagset 'v5'
• Add new hashes with more consistent naming, and in some cases, easier and more consistent data layout.
• Add subs to provide an easier interface to the hashes.
• Add ability to export subs and hashes.
• Encourage people to use new subs and hashes.
• We only support HTML4 or HTML5 at one time. You cannot do use HTML::Tagset ( 'v4', 'v5' );.

Existing HTML4 support

Existing code that uses HTML::Tagset without parameters will behave identically. You will get the same hashes you always have.

use HTML::Tagset;
my $is_head = $HTML::Tagset::isHeadElement{ $tag };

# Specifying 'v4' is identical.
use HTML::Tagset 'v4';
my $is_head = $HTML::Tagset::isHeadElement{ $tag };

You will also be able to use newly-named hashes and subs. All new names will be lowercase-and-underscores, and begin with tag_.

my $is_head = $HTML::Tagset::tag_is_head_element{ $tag };
my $is_head = HTML::Tagset::tag_is_head_element( $tag );

The newly-named hashes are NOT necessarily identical to the old hashes.
For example, the existing %linkElements looks like this:

our %linkElements = ( 'a' => ['href'], 'applet' => ['archive', 'codebase', 'code'] ... );

The newly-named hash %tag_is_link_attribute's values will be hashrefs. Same data, different layout.

our %tag_is_link_attribute = ( a => { href => 1 }, applet => { archive => 1, codebase => 1, code => 1 }, ... );

The new hashes will be easier to use.

# The old way.
my $linkElements = %HTML::Tagset::linkElements{ $tag };
if ( $linkElements ) {
    my %h = map { $_ => 1 } @{$linkElements};
    $is_link_attribute = $h{ $attr };
}

# The new way
my $is_link_attribute = $HTML::Tagset::tag_is_link_attribute{ $tag }{ $attr };
my $is_link_attribute = HTML::Tagset::tag_is_link_attribute( $tag, $attr );

HTML5 support

You can specify 'v5' to get HTML5-based data.

use HTML::Tagset 'v5';
# Same as 'v4' but with v5 hashes and subs.
my $is_head = $HTML::Tagset::isHeadElement{ $tag };
my $is_head = $HTML::Tagset::tag_is_head_element{ $tag };
my $is_head = HTML::Tagset::tag_is_head_element( $tag );

You will also be able to export hashes and subs.

use HTML::Tagset qw( v4 :hashes );
use HTML::Tagset qw( v4 :subs );
use HTML::Tagset qw( v5 :all );
my $is_head = tag_is_head( $tag );

If you export, you MUST pass a version. You may not pass multiple versions.

use HTML::Tagset qw( :all );    # Fails, no version
use HTML::Tagset qw( v4 v5 );   # Fails, too many versions

Disallow use of HTML::Tagset::* namespace

There will be no way to support both v4 and v5 at the same time.

Behind the scenes, the HTML::Tagset hashes and subs will probably be aliases to their ::v4 or ::v5 counterparts. I don't think we want to support people referring to the version-specific namespaces directly. They are only there for ease of development.

# YES
use HTML::Tagset 'v5';
my $is_head = HTML::Tagset::tag_is_head_element( $tag );
my $is_head = tag_is_head_element( $tag );
# NO, not guaranteed to be supported in the future.
my $is_head = HTML::Tagset::v5::tag_is_head_element( $tag );

[petdance]

Also: We can implement this new interface on the existing v4 code, make sure it works, set up new tests, release it as HTML::Tagset v3.xx, etc before we even touch going to support HTML5 in HTML::Tagset v5.00. Two manageable chunks rather than one giant overhaul.

[PhilterPaper]

I'm OK with leaving the current hashes alone, as (completed set) HTML 4, and support adding new functionality (block elements, attributes, parent-child relationships, HTML 5, etc.) as subs (methods). The default for these would be HTML 5, and might include additional flags to include/exclude deprecated, withdrawn, or niche tags/attributes. I think most users would be interested in defaulting to HTML 5, but would be able to tailor the lists to specific HTML levels if they're doing something requiring a specific tag set.

One of the nice things about using a sub is it's cleaner to use code to build up a list, starting with, say, HTML 3.2, adding tags or elements for HTML 4.01, and finally adding to the list for XHTML 1.0 and HTML 5. Also, build up lists from lower levels to high: all_allowed_elements() pulls in all_head_elements() and all_body_elements() and both_head_and_body_elements(), all_body_elements() pulls in block_elements() and flow_elements() and anything else that's body-only, block_elements() pulls in (among other things) table_elements(), etc. is_block_element($tag) could call block_elements() to get the block list, and check that one element. The idea is that there should be no duplication of effort and possible inconsistencies with manually-maintained lists. Each higher-level call would simply pass on the same arguments (@_) to lower-level calls.

Anyway, that's the way I would approach the problem. It would be a good opportunity to clean up the various list names for consistency. We could provide subs to provide a list of tags (elements) at each level (all_XXXX()) and check if a given tag is in that list (is_XXXX($tag)). It remains open whether an "all_" list would be an array or hash; "is_" returns a boolean.

My two cents worth.

[petdance]

I want to abandon the old-style hashes entirely. I don't want to bring them forward to HTML5. They are inconsistent and I don't want to carry that forward.

New principles I'm adding to the list:

• HTML::Tagset will run on Perl 5.10.1 and up
• It will not use any non-core modules. Exporter and List::Util and Test::More should be all it needs.
• We want people to migrate off of old-style hashes. They are there only for backward compatibility.
• The old style hashes will stay updated in v4. They will NOT be available for v5.
• Any new hashes created, even for v4, will not include an old-style hash.

[petdance]

This is what I see as needed for our first step to add new hashes while keeping the old hashes in place. We can release HTML::Tagset 4.00 with these new hashes and subs in place. It will be backwards-compatible with 3.24 and let people start using the new hashes and subs while we work on HTML::Tagset 5.00.

Old and new hashes and functions

All existing hashes will not change.

All new names will be all lowercase-and-underscores.

Boolean lookups will be named tag_{boolean_phrase} e.g. tag_is_known.

Lists will be named tags_{plural_phrase} e.g. tags_closure_barriers.

Hash values will always be a boolean, or a hashref of other booleans.

New functions will be wrappers around the hashes.

# Old
our %emptyElement   = map { $_ => 1 } qw( base link meta isindex ... );
# New
our %tag_is_empty_element;
sub tag_is_empty_element( $tag );

# Old
our %optionalEndTag = map { $_ => 1 } qw( p li dt dd); # Add option th tr td);
# New
our %tag_end_is_optional;
sub tag_end_is_optional( $tag );

# Old: Values are arrayrefs.
our %linkElements = ( 'a' => ['href'], 'applet' => ['archive', 'codebase', 'code'] ... );
# New: Values are hashrefs.
our %tag_link_attributes = ( a => { href => 1 }, applet => { archive => 1, codebase => 1, code => 1 }, ... );
sub tag_link_attributes( $tag );
sub tag_attribute_is_link( $tag, $attr );

# Old: Values are scalar if there is only one attribute, or hashrefs if there is more than one.
our %boolean_attr = ( 'area' => 'nohref', 'input' => { 'checked' => 1, 'readonly' => 1, 'disabled' => 1 }, ... );
# New: Values are hashrefs.
our %tag_boolean_attributes = ( area => { nohref => 1 }, 'input' => { 'checked' => 1, 'readonly' => 1, 'disabled' => 1 }, ... );
sub tag_boolean_attributes( $tag );
sub tag_attribute_is_boolean( $tag, $attr );

# Old
our %isPhraseMarkup = map { $_ => 1 } qw( span abbr acronym q sub sup ... );
# New
our %tag_is_phrasal;
sub tag_is_phrasal($tag);

# Old
our %is_Possible_Strict_P_Content = ( ... );
# New
our %tag_is_possible_strict_p_content;
sub tag_is_possible_strict_p_content( $tag );

# Old
our %isHeadElement;
# New
our %tag_is_head_element;
sub tag_is_head_element( $tag );

# Old
our %isList;
# New
our %tag_is_list;
sub tag_is_list( $tag );

# Old
our %isTableElement;
# New
our %tag_is_table_element;
sub tag_is_table_element( $tag );

# Old
our %isFormElement;
# New
our %tag_is_form_element;
sub tag_is_form_element( $tag );

# Old
our %isBodyElement;
# New
our %tag_is_body_element;
sub tag_is_body_element( $tag );

# Old
our %isHeadOrBodyElement;
# New
our %tag_is_head_or_body_element;
sub tag_is_head_or_body_element( $tag );

# Old
our %isKnown;
# New
our %tag_is_known;
sub tag_is_known( $tag );

# Old
our %canTighten;
# New
our %tag_can_tighten;
sub tag_can_tighten( $tag );

# Old
our @p_closure_barriers;
# New
our @tags_p_closure_barriers;
sub tags_p_closure_barriers();

# Old
our %isCDATA_Parent;
# New
our %tag_is_cdata_parent;
sub tag_is_cdata_parent( $tag );

[PhilterPaper]

The problem I have with separate v4.00 and v5.00 as you propose is that there is bound to be confusion for users as to which to use, and what version is updated automatically by tools such as cpan. Therefore, my proposal to leave the existing hashes alone (same name and structure), updated to full HTML 4, under version 4.0. Add new subs (methods) with rational names to allow selection by HTML version in release 4.1 (if not 4.0). A 5.0 release number would be OK.

The end result is that some system blindly updating HTML::Tagset "to the latest and greatest" will get a system which is still perfectly compatible with the current system, and doesn't require packages using HTML::Tagset to immediately rewrite their code (to use no higher than 4.00). All the new functionality (HTML version selection) and rational names becomes immediately available for anyone who updates their code to use the new library. Packaging system such as cpan can then safely update existing HTML::Tagset packages to 4.0 or beyond without fear of breaking applications.

If your plan is to release a separate HTML5::Tagset, backwards incompatible with 3.24, I don't see the point of that (now there are two Tagsets to maintain). If it remains in the same HTML::Tagset namespace, but v5.00 is planned to break all existing applications, that's not good. If 3.2 or 4.0 is currently installed, and HTML::Tagset 5.0 is released, most update systems (such as cpan) will automatically update to 5.0, breaking applications that haven't been updated to use the new capabilities. Am I missing something in your plans?

[petdance]

I'm sorry I wasn't clear. Here's what I see as the plan.

Right now we have HTML::Tagset 3.24 with the old hashes.

use HTML::Tagset;

Next comes HTML::Tagset 4.00 which will have the old hashes and the new hashes and functions. It will still be HTML4 only.

use HTML::Tagset;
use HTML::Tagset qw( v4 :hashes );
use HTML::Tagset qw( v4 :subs );
use HTML::Tagset qw( v4 :all );

Then comes HTML::Tagset 5.00 which will have everything that 4.00 has, plus the support for HTML5.

use HTML::Tagset qw( v5 :hashes );
use HTML::Tagset qw( v4 :subs );
# etc

There will probably only be one or two releases of the 4.x line before we move forward to the 5.x line. The 4.00 release will get a version bump to alert folks that it is a major update, even though it does not yet handle HTML5.

The end result is that some system blindly updating HTML::Tagset "to the latest and greatest" will get a system which is still perfectly compatible with the current system, and doesn't require packages using HTML::Tagset to immediately rewrite their code (to use no higher than 4.00). All the new functionality (HTML version selection) and rational names becomes immediately available for anyone who updates their code to use the new library.

Yes, that is exactly the plan.

If your plan is to release a separate HTML5::Tagset, backwards incompatible with 3.24,

No, not at all. There will not be separate modules. Someone running HTML::Tagset 3.24 with standard use HTML::Tagset; will be able to upgrade to 4.00 and/or 5.00 with no changes to their code.

[PhilterPaper]

OK, let's say I'm running a poorly-supported application (such as HTML::Tree) that makes use of HTML::Tagset (currently 3.24) for its list of tags (admittedly not quite up to date for HTML 4). I use cpan to daily update my Perl packages. You release HTML::Tagset 4.00, which cpan automatically upgrades me to -- so far, so good, it's compatible with the current 3.24. The application does NOT get updated, and some time after, you release HTML::Tagset 5.00. cpan will automatically upgrade me to 5.00 -- and does the application now break? If so, it sounds like you're counting on every application using HTML::Tagset to get promptly updated to use the new format (with HTML version specification). Or is it actually backwards compatible if you just continue to say use HTML::Tagset; without the V4/V5 flag? Does that now default to V5? What you're saying seems to have changed in your latest clarification, and I'm confused. My specific usage of HTML::TreeBuilder should be happy to supply HTML 5 tags to me (as long as the hash names haven't changed), but it is possible that some applications and users really are counting on it staying HTML 4.

Is this your intent:

v 3.24 release
  use HTML::Tagset;                provides "traditional" hash variables (with incomplete HTML 4)

v 4.00 release
  use HTML::Tagset;                provides "traditional" hash variables (with complete HTML 4)?
  use HTML::Tagset qw( V4 ... );   provides NEW hashes (only) and subs?

v 5.00 release
  use HTML::Tagset;                provides "traditional" hash variables? still HTML 4?
  use HTML::Tagset qw( V3 ... );   provides NEW hashes (only) and subs at HTML 3?
  use HTML::Tagset qw( V4 ... );   provides NEW hashes (only) and subs at HTML 4?
  use HTML::Tagset qw( V5 ... );   provides NEW hashes (only) and subs at HTML 5?

[petdance]

some time after, you release HTML::Tagset 5.00. cpan will automatically upgrade me to 5.00 -- and does the application now break?

As I said above: "Someone running HTML::Tagset 3.24 with standard use HTML::Tagset; will be able to upgrade to 4.00 and/or 5.00 with no changes to their code."

use HTML::Tagset; will always default to HTML4.

HTML::Tagset 4.00 will provide everything 3.24 does, AND it will provide new hashes and subs.

HTML::Tagset 5.00 will provide everything 4.00 does, AND it will provide a choice of HTML4 or HTML5.

I have no interest in supporting older versions of HTML.

[haarg]

If a user has to request v5, what is the advantage of using an import option rather than just having a separate module? With a v5 import option, it will require a more complex import method. And it would either require extra magic in the subs or they would not be able to be called fully qualified. It seems like it would make more sense to just make it a separate module.

[petdance]

it will require a more complex import method.

At this point I'm not worried about code internals. We're just talking about interface.

But you've raised a good question.

To @PhilterPaper and everyone else: Why should we update HTML::Tagset to do HTML 5 vs. creating, say, HTML::TagWhatever that would be HTML5 only, and entirely incompatible with HTML::Tagset?

Do we need to have a module that can handle both HTML4 and HTML5? Why? What's the use case?

[oalders]

To @PhilterPaper and everyone else: Why should we update HTML::Tagset to do HTML 5 vs. creating, say, HTML::TagWhatever that would be HTML5 only, and entirely incompatible with HTML::Tagset?

It seems to me that it would be a lot simpler to release HTML5::Tagset (or whatever name is appropriate) and move ahead from there.

[petdance]

I wouldn't want make "HTML5::" a top-level namespace. HTML::Tagset5 is what's been thrown around before.

It seems to me that it would be a lot simpler

Yes, for the maintainers of HTML::Tagset. I also don't know what the needs of the folks downstream are. That's why I'm asking from input there.

[PhilterPaper]

I'm good with keeping HTML::Tagset as the top-level namespace, and either using qw() on the 'use' or (better, IMO) parameters/options on sub (method) calls to select HTML version. Whatever is done, a bare HTML::Tagset should be HTML 4 (full set) existing hash names so it doesn't break existing applications (which might not get updated for a long time). Add on other stuff to enhance its function.

Don't forget that there was an XHTML; I don't recall offhand if it introduced new tags or attributes, but it did change the rules on a number of things (self-closing tags, quotes on attributes, etc.).