November 2010 Archives

Thoughts about Mutli-Selection on the Modern Web

We have several applications where we need to let the user select multiple items. HTML has pretty much always offered multi-select through it’s forms in a fairly simple manner, complete with how it will be rendered on screen:

Multiselect.png

<p>
   <label for="fruit">Select a fruit</label>
   <select name="fruit" size=4 multiple>
    <option value="apple">Apple</option>
    <option value="apricots">Apricots</option>
    <option value="banana">Banana</option>
    <option value="cherries">Cherries</option>
    <option value="durian">Durian</option>
    <option value="grapes">Grapes</option>
    <option value="kiwi">Kiwi</option>
    <option value="kumquat">Kumquat</option>
    <option value="mango">Mango</option>
    <option value="strawberry">Strawberry</option>
   </select>
</p>

It’s a simple bit of UI, but the fact is, most users have no idea how to use it. We have a multi-select box that is populated with rows that the user would generally have need to access. For most users, there are a half-dozen or so of these rows. Other users, can receive an authorization that provides them dramatically more rows. When we first launched this UI, which was a change from the old system of select boxes that did not make it clear what data the user was limited to under the stricter interpretation of the law that we were using.

The change, however, made selecting multiple records seem harder to our users. They simply didn’t know that SHIFT+Click to select ranges or CTRL+Click (Command+Click for you Mac users) to select discrete items. Now, some sites will drop in instructions to help users figure out how to multiselect, but as our user base was relatively small (particularly who demanded multi-select) we proceeded without it.

However, I have recently re-launched Washington State University’s Schedules of Classes site, and in re-doing the search form, I felt I needed a better way, particularly since one of the select boxes has ~150 items. To deal with that, I utilized a pair of select boxes and server-side sessions to allow the user to select one (or many) items from the ‘source’ list on the left to the destination list on the right.

Option Mover

This works okay, and I still use it for progressive enhancement. It should eliminate the problem of users not knowing how to select many options, though that remains to be seen, and I have seen it elsewhere. I think this makes a reasonable amount of sense in situations like this, where the number of elements that will be expected in the right column will be fairly small.

For a more involved solution, I opted to investigate re-inventing multi-select boxes, by looking at how they are done in Mobile. On Android, a mutli-select box causes a modal dialog to appear, where the select items are represented by checkboxes (it uses radio buttons for single-select). Now, a modal dialog is wholly unnecessary on the desktop, but the checkbox option makes a lot of sense.

To support this, I wrote a YUI module (which I’m investigating releasing through the Gallery, there are copyright issues with my employer, though), which is in use on that page. It is a widget, which is attached to an element containing a label, and an input[type=select] element (at least one, it will ignore any more that it sees).

Checkbox List

The markup generated looks like this:

<p class="yui3-form-checkboxlist>
   <label for="fruit">Select a fruit</label>
   <ul>
    <li><input type="checkbox" name="fruit" value="apple">Apple</input></li>
    <li><input type="checkbox" name="fruit" value="apricots">Apricots</input></li>
    <li><input type="checkbox" name="fruit" value="banana">Banana</input></li>
    <li><input type="checkbox" name="fruit" value="cherries">Cherries</input></li>
    <li><input type="checkbox" name="fruit" value="durian">Durian</input></li>
    <li><input type="checkbox" name="fruit" value="grapes">Grapes</input></li>
    <li><input type="checkbox" name="fruit" value="kiwi">Kiwi</input></li>
    <li><input type="checkbox" name="fruit" value="kumquat">Kumquat</input></li>
    <li><input type="checkbox" name="fruit" value="mango">Mango</input></li>
    <li><input type="checkbox" name="fruit" value="strawberry">Strawberry</input></li>
   </ul>
</p>
.yui3-form-checkboxlist > ul
{
   overflow: hidden;
   list-style-type: none;
}

It’s more verbose, but in a progressive enhancement situation that isn’t necessarily a problem, but more importantly, the form output that comes out of this when POSTing matches what comes out of the original implementation, so whatever you use to get the selected items on the server for a mutli-select should work here as well. I am not sure I have exceptional accessibility on this widget yet, but it should be possible to do fairly easily.

The more I look at this, the more I kind of question if I shouldn’t have just served up the checkbox list from the server. I suppose it’s a question for me of accessibility, particularly without JavaScript enabled, but I suppose the implementation I am enhancing may not be as accessible as it could be, even with title’s set to describe what the buttons do. Still, the concept of replacing multi-select boxes with lists of checkboxes makes a lot of sense to me, though it’s not as semantic for those browsers that already render in this fashion.

Codification of Common Courtesy

The Open Source community had a pretty sad movement start last week, in the form of the Open Respect Declaration started by Jono Bacon, but plenty of others have gotten involved. And the Decleration is certainly not bad, just it seems as if it’s the kind of thing that should be wholly unnecessary.

I understand that it is not. Noirin Shirley posted on her blog about an unfortunate and abhorrent experience she had at this year’s ApacheCon in which she was sexually assaulted by another member of that community. What’s worse, the comments on her blog have more than a few where people feel she was in the wrong for calling out the member who assaulted her.

Now, I know that Open Respect is about much more than simply the misogyny present in the Open Source (or any Technical) field, or gaming and other traditionally male-dominated sub-culture. As a group, we tend to dismiss out of hand any opinion that runs contrary to our own, usually to the exclusion of any real discussion, let alone reasoned debate.

The Open Respect Declaration is, quite simply, a request that people behave reasonably. To disagree in a manner that doesn’t insult one another, in spite of the strength of conviction on our opinions. To behave as the meritocracy that we have always claimed to belong to in tech.

Last week, I was at YUIConf, held on the Yahoo! Campus. While there, I spent significant amounts of time with people from Brazil, met dozens of Indians and Middle Easterners, and yes, even an (admittedly small) number of women. I believe I conducted myself in a respectful manner, and I am aware of no real problems with serious disrespect which occured at this event. However, that is simply my perception, and while I am reasonably observant, I am also not one whom is generally subjected to disrespect.

We are a sub-culture which is full of holy wars. vi-vs-emacs. Unix-vs-Windows. jQuery-vs-Everything Else. And I am certainly more opinionated than most on these subjects. I can, at times, be somewhat abrupt in my discussion, but I do strive to at least allow for the other person to be wrong. Or right. I’m argumentative, but it’s not really about winning, it’s about the debate. Sometimes it’s unsuccessful and unproductive, and I’m constantly working on making the debate more approachable for those people who shy away from anything resembling conflict.

Ultimately, Open Respect is something that should apply to any situation, not just Open Source and other technology fields, and hopefully this little movement that’s started can help make those members of our community who, intentionally or otherwise, behave hostilely or rudely, recognize their behavior, and begin to correct it.

ASP.NET MVC and AJAX Updating

In the work I’ve been doing to update Washington State University’s online Schedules of Classes, I wanted to support users who had access to HTML5 History, by letting them save time on page loads by only bringing back the part of the page where content was actually changing for a fair number of clicks that would happen on this page.

Now, the utility I use to accomplish this in JavaScript, YUI3’s History module would support, more or less transparently, using the Hash to let users who aren’t on HTML5 compatible browsers utilize this feature, but that would have broken my URL bar, adding complexity to the use case of copying and pasting the URI into an e-mail or whatever else to share with other users.

With MVC, I knew that I’d be able to support the exact same URIs for the pages that were requested via AJAX as those requested via a more normal usage pattern. However, I also need to be sure that, when requested via AJAX, I use a different master page for the serving.

On my pages, there are two Content Areas on the page of interest. Fist, MainContnt is the inside of a div with the id of ‘main’ on my Site.master. Second, ScriptContent is at the bottom of the page, so I can optionally add JavaScript content to be served on a page.

First, you’ll need to define you new Master page. I named mine AJAX.master, and it appears as follows:

<@ Master Language="C#" AutoEventWireup="false" %>
<asp:ContentPlaceHolder ID="MainContent" runat="server">
</asp:ContentPlaceHolder>
<asp:ContentPlaceHolder ID="ScriptContent" visibility="false" runat="server">
</asp:ContentPlaceHolder>

Visual Studio will complain about this not being valid, since it isn’t valid XHTML, but it’s a warning that, for this at least, you’ll just need to accept. Also, very important, is that visibilty="false" attribute on the “ScriptContent” placeholder. While declaring a ContentArea in a MasterPage does not require it to appear in a View, if you use it in your views, you will require it on the MasterPage, but with the visibility set to false, it won’t actually be rendered to the wire.

The JavaScript to make this request looks like this:

YUI().use('io-base', 'node', function(Y) {
    var show = function(node) {
        node.setStyle('display', 'block');  
    }, hide = function(node) {
        node.setStyle('display', 'none');
    },
    sessionTimestamp = (new Date()).getTime(),
    mainContent, loadingContainer, dataContainer;

    mainContent = Y.one('#main');
    dataContainer = Y.Node.create('<div></div>');
    loadingContainer = Y.Node.create('<div style="display: none;"><img src="/loading.gif" /></div>');

    mainContent.setContent("");
    mainContent.append(dataContainer);
    mainContent.append(loadingContainer);

    Y.io("/uri/to/request", {
        data: "ts=" + sessionTimestamp,
        on: {
            start: function (id) {
                show(loadingContainer);
                hide(dataContainer);
            },
            success: function (id, o) {
                dataContainer.setContent(o.responseText);
            },
            failure: function (id, o) {
                var error = dataContainer.one('p.error');
                if (!error) {
                    dataContainer.prepend("<p class='error'></p>");
                    error = dataContainer.one('p.error');
                }
                error.setContent("There was an error retrieving your search results. Our staff has been notified.");
            },
            end: function (id) {
                show(dataContainer);
                hide(loadingContainer);
            }
        }
    });
});

The above modified #main so that we can show a loading image, I generate mine onn ajaxload.info, while we load the new page. Now, in truth, this is wrapped in a delegate event handler attached to all the ‘a’ links on the page, and a check to make sure it’s one that should be handled in this way, but I’m trying to simplify as much as possible.

Okay, so we have our AJAX.master set up, and the JavaScript written, but that doesn’t make my Views work correctly. Luckily, there is a de facto standard in AJAX libraries these days that they all set the HTTP Header “X-Requested-With” to “XMLHttpRequest” when making AJAX calls. I know that YUI and jQuery both do this, I’m fairly certain that Prototype, MooTools and Dojo do as well.

By adding the following class, you can avoid needing to perform this check inside of each individual controller action method.

namespace foxxtrot.MVC.AJAX {
    using System.Web.Mvc;

    public class AJAXViewEngine : WebFormViewEngine {
        public override ViewEngineResult FindView(ControllerContext context,
                                                  string viewName,
                                                  string masterName,
                                                  bool useCache)
        {
            if (context.HttpContext.Request.Headers["X-Requested-With"] == "XMLHttpRequest") {
                masterName = "AJAX";
            }

            return base.FindView(context, viewName, masterName, useCache);
        }
    }
}

It’s that simple. Now, anytime one of my Controllers gets called with an AJAX-y header, it will return back only the MainContent. To activate it, just add the following lines to your Application_Start method in your Global.asax.cs file.

ViewEngines.Engines.Clear();
ViewEngines.Engines.Add(new foxxtrot.MVC.AJAX.AJAXViewEngine());

Soon I’ll have time to finish speeding up my database layer, so you’ll be able to experience this technique on a live site.

Meet me at YUIConf 2010!

For the next sevral days I’m going to be a YUIConf, Yahoo!’s Front-End Engineering Conference. I expect I’ll have plenty of stuff to talk about here after the event, particularly Node.JS related, as I intend to avail myself heavily of that track.

For those who are here as well, I look forward to meeting you! For those who are not, the sessions will be posted online sometime soon after the conference. I’ve been here in the Bay Area since Saturday, and my only regret now is that I leave pretty early on Thursday, so I won’t have much time for exploration.

Thoughts on YUI vs. jQuery

There have been several discussions about how YUI can better compete in the JavaScript Framework space. The biggest discussion has been held on quora in a thread entitled How could YUI improve its image compared to jQuery, MooTools, etc.. The question garnered enough attention that John Resig, creator of jQuery, the de facto leader in this space, felt inclined to respond.

John’s comments begin fairly reasonably. The YUI project does need to centralize on a single location. Currently, the forums, gallery and bug trackers are on YUI Library, while the core documentation still lives on the [Yahoo! Developer Network (http://developer.yahoo.com/yui/). However, this is already something that is being addressed, as demoed by Allen Rabinovich at YUI Open Hours from September 29th (video), and it’s an issue I know I raised back when YUI Library was launched.

More than that, the redesign of YUI Library really seems to be centered around highlighting the gallery, in effect trying to show off gallery modules as nearly on par with the core library (though still filterable and clearly marked). Which goes a long way to address the issue of making community contributions clear. Plus, the core team has been getting better at looking at external submissions, though there are a few places I’d still like to see them improve in that respect, and no, I’m not talking about my builder agenda.

Now, for the rest, I’m going to try hard to to parrot Nicholas Zakas’ comments, though I agree with pretty much everything he has to say. The very idea that YUI’s association with Yahoo! is a weakness is simply ridiculous.

Admittedly, there are a lot of open source projects which are corporately sponsored that are hampered by that association. MySQL’s development didn’t seem as rich after Sun bought it, and now Oracle is reducing what the open source edition is capable of. Microsoft has a lot of MS-PL code that they can’t take external contributions for, making their code drops nearly useless. Google’s Android is much the same way, since the work being done by Google Engineers only seems to be made public in periodic large code drops.

Particularly through Gallery, YUI has gotten more and more open to external contributions. Personally, I’ve even gotten a few small changes in the core of 3. What makes an Open Source project is not it’s sponsors, but it’s community. It’s why Canonical has been receiving so much flak lately from the GNOME Community. They aren’t playing nice with the rest of the community, in part because they seem to think that innovation can’t happen quickly enough in that environment. I’m split on the issue, but that’s not the subject of this post.

Yahoo! has built a solid community around YUI. And that community has gotten a lot stronger in the last year. Just looking at the gallery shows a healthy number of contributions, most from outside of Yahoo! While the core is mainly commits from inside, they aren’t all, but that’s the way of most Open Source projects. Most commits come from a small core of developers. And Nicholas is right, having YUI inside of Yahoo! drives some pretty awesome development. The core modules for the Rich Text Editor and Autocomplete for YUI3 were driven by internal needs, that will now benefit all of us.

Personally, I don’t view YUI and jQuery in the same space. jQuery has always felt to me like a DSL for the DOM. A person could use it just fine without knowing any JavaScript. And the people I know who love it most, seem to fall into that mold. YUI3 requires at least basic skill with the language, and more if you really want to exploit it well and extend it. There are some serious web sites built with YUI, and it’s excelled at that. I suspect that more people will begin to take note of that sooner rather than later, especially if we can improve the documentation that introduces a user to the library.

It’s not an us versus them argument, and couching it in those terms is going to do us a lot more harm than good.

Paper Review: Thumbs Up or Thumbs Down?

Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews, a 2002 paper from Peter D. Turney of Canada’s National Research Council, was referenced by the Semantic Analysis paper I read a few weeks back.

You may recall that I quite enjoyed that paper, and I felt like I was getting a fair amount out of it. However, I feel that about all this paper really did for me was point me toward other papers that may contain more real information.

The algorithm proposed is interesting, in that it looks at adjectives and adverbs to determine tone, but attempts to assign context by taking the next word, since the context of a adjective can determine if it’s positive or negative sometimes. This measurement is done based on a statistical ratio weighting on probabilities that words co-occur.

This idea of phrase dependence on the adverb/adjective does appear to carry some success. On non-film reviews, the algorithm would generally be able to guess over 70% of the time if the review was positive or negative. Even with film, it was right 60% of the time, only weakened by the fact that almost every movie carries with it good and bad that people talk about in the same review.

I’m not entirely sure how he seeded the values used to generate the Semantic Orientation scores, which is probably why I’m not terribly into this paper. Those scores appear to hold the majority of sway over the outcome of the algorithm, and I don’t know where they came from.

The basic finding, that you get a significant boost to the accuracy of your result by considering the word following the adjective or adverb, is significant, but for some reason this paper really failed to grab my attention with the possibilities it raised. Perhaps I just need to read more of the work it’s based on.

Next Week: On sequential Monte Carlo sampling methods for Bayesian filtering

Getting the Most out of LINQ

Language Integrated Queries, or LINQ, is a language feature Microsoft introduced into .NET 3.5 which allows for a more functional approach to programming in .NET in that it allows you to easily work on collections of data, filtering them with custom operations, transforming them (mapping), or combining them (reducing) in interesting ways. I use LINQ all the time in my C# code these days, however, I’ve found that more and more, I’m using the so-called ‘Lambda’ syntax over the ‘query’ syntax.

The Query syntax is meant to be simpler, but that simplicity makes it far less powerful. But to start, here’s a pair of identical queries, first in Query syntax, then Lambda. Note, I’m talking here about LINQ-to-Objects, but this translates to LINQ-to-SQL almost as easily.

var collection; // Some sort of collection, anything that implements IEnumberable

var queryResult = from c in collection
                  where c.value == "Awesome"
                  select c.otherField;

var lambdaResult = collection.Where(c => c.value == "Awesome")
                             .Select(c => c.otherField);

Two identical queries, with a collection being filtered and then mapped to a new imput. Now, I do find that Query syntax is far easier to map with than lambda syntax, so I typically start my queries in query syntax, to get the basics out of the way, and in the background the query syntax is converted to Lamda syntax before compilation, making these examples nearly identical in their bytecode output. The point of the Query syntax, I believe, was to be more familiar for those people accustomed to know SQL, but aren’t comfortable with functional programming, however, using Lambdas in C# allows you to do some very powerful and useful things.

To take brief sidetrack into LINQ-to-SQL (or Entities, this would work either way), I use code similar to the following for a lot of queries I run in my data models.

public class Model {
    var database = new DatabaseContext();

    private IQueryable<LocalDataType> GetBaseQuery()
    {
        return from d in database.Table
               select new LocalDataType()
               {
                    Id = d.Id
                    Value = d.field1,
                    Data = d.field2
               };
    }
}

LocalDataType is simply a Data Transfer Object (DTO) that I’ve defined as what needs to be returned from my Model. By having GetBaseQuery private, I can publish an interface of functions that know how to get the data out of database.Table, and translate it into LocalDataType, all while being able to apply their own filters, with virtually no code dupication. For instance, to get the data based on it’s numeric Id, I can add the following function:

private IEnumerable<LocalDataType> GetById(int id)
{
    return GetBaseQuery().Where(c => c.Id == id);
}

Let’s say I have more complex filtering, where I may not always need to test something:

private IEnumerable<LocalDataType> GetByCode(string[] validCodes)
{
    if (validCodes.Length > 0) {
        return GetBaseQuery().Where(c => validCodes.Contains(c.Data));
    } else {
        return GetBaseQuery();
    }
}

This allows me to quickly build variable queries that, in the case of LINQ-to-SQL, will always execute on the SQL Server. SQL is good at filtering, particularly over keyed values, so you’re always going to have better performance doing as much filtering on the SQL Server, even before you count the reduced data coming over the wire.

My point is simple: the Query syntax of LINQ is a bit misleading. It suggests that queries are things that are built once and used as they are. However, they are in fact mercurial things, which can be built on and extended and made better as you work with them, and seeing that is a lot clearer in the Lambda syntax. Using LINQ in this fashion will make your code far more maintainable and easier to extend, since you’ll be working with common building blocks, and you’ll be avoiding that copy-paste mentality that can ofter pervade hard-to-maintain codebases.

Plus, there are some functions, like Aggregate, that don’t exist in Query syntax. In my current project, I need to occasionally filter by course level, ie 100-level, 200-level, etc. The easiest way to do this, it turns out, is to test the first character of each course against a list of approved courses. In addition, there is a requirement that one choice, actually maps out to all 600-, 700-, and 800- level courses.

private IEnumerable<LocalDataType> GetByCourseLevel(IEnumerable<int> courseLevel)
{
    //Course Level is a list of ints 100, 200, 300, 400, etc, fed in based on the selections off a HTML form.
    string courseLevelWorking = courseLevel == null ? string.Empty : courseLevel.Aggregate<int, string>(string.Empty, (workingString, next) => {
        if (next == 600) {
            return workingString + "678";
        } else {
            return workingString + (next / 100).ToString();
        }
    });

    var query = GetBaseQuery();
    if (courseLevelWorking != string.Empty) {
        query = query.Where(c => courseLevelWorking.Contains(c.CourseNumber.ToString()[0]));
    }
    return query;
}

LINQ allows you to build amazingly complex queries in an additive and clean fashion, and taking your LINQ usage from simple, to complex yet maintainable requires very little work, with just a bit of forethought.

Palouse Code Camp 2010 Wrap-Up

This Saturday was the first ever Palouse Code Camp, hopefully the first is a long line. We’d been planning it seriously for 8 months or so, though we’d been knocking the idea around for over a year. We did not draw the crowd we’d hoped for (we had ~30 people), so it was a very small event, but those who attended seemed to enjoy the event, so I think it still needs to be counted a success. Our largest failure in advertising was clearly with the students, as we had virtually no student representation, something which we’ve identified ways to fix for next year. Our sponsorship was also dramatically lower than we’d hoped, but generous donations from Microsoft and WSU’s Social & Economic Sciences Research Center ensured that all our expenses for the year were covered, even leaving us a bit left over to keep us afloat until we start fundraising for next year (which will start much sooner).

We really appreciated all our speakers, Jack Stephens from Spokane, gave a sort of overview of LINQ, using part of it from his talk on using LINQ with DataSets. Dave Sargent, an organizer, who talked about Website Performance (based in part on a talk I gave a few years ago) and MS SQL Server Administration. WSU Professor Robert Lewis with is Introduction to Python talk, which I think has finally convinced Catherine that she really ought to learn Python for her research work. Mithun Dhar, our regional Developer Evangelist from Microsoft, came out to talk about some of what Microsoft is doing in the near term, and to give all our attendees a free month of Windows Azure service. Jason Hurdlow, organizer, redid his XSLT talk, focusing this time a bit more on XPath, I think.

However, I want to give a very special thanks to Mark Michaelis, who volunteered to do “as many talks as we needed”, and gladly gave us four. MSBuild, Powershell, MVVM with WPF, and Pragmatic Unit Testing. I didn’t get a chance to attend any fo Mark’s talks, but I do believe they were very well attended. We’d only made contact with Mark a bit over a week from the event, and his support was amazing.

Myself, I gave two talks. The first, was an update of my Introduction to YUI3 talk I gave at Portland Code Camp earlier this year, updated for YUI 3.2.0 (and of course, YUI 3.3.0pr1 was tagged in git today). I had around ten people, and amazing turnout given the size of the event, and was ecstatic. This talk focused more on SimpleYUI, but I made sure to touch on the Module pattern, as I’m still not entirely comfortable with SimpleYUI. The code is up on GitHub, and the slides are on SlideShare. For those who attended the talk, I’d really appreciate any ratings you can provide.

My second talk was about the YUI3 Component Framework and module creation, and it only had a single attendee, but he was willing to stay, and I wanted to talk about it, so I went ahead. Slides here.

We learned a lot to improve for next year, and a lot of the groundwork is done, so I fully expect to have a good success in 2011.