Alt Text Bot Image Descriptions FTW

This weekend I saw a tweet in Marcy Sutton’s timeline that appeared to be an image description generated by a piece of software.

Given my recent missives on the inherent inaccessibility of images without descriptions (even if Twitter accidentally gave us more options), coupled with rise in people tweeting images of text to get around character limits, I was intrigued.

It turns out the Alt Text Bot is a project from Cameron Cundiff that he submitted to the NYU ABILITY Technology Hackathon, where it also won first place this weekend. He has written a little bit of background on the bot.

Alt_Text_Bot uses an API from CloudSight to help describe images submitted in tweets. Users simply need to mention @alt_text_bot in a tweet with an image (the tweet must be part of the image, not in a Twitter card or via a link) and Alt Text Bot will respond with a description.

I’ve been feeding my own test images, but Steve Faulkner has been testing its ability to read CAPTCHAs and recognize faces of personalities (though not all).

It has some limitations. The biggest is the character limit within Twitter. Converting a chart to text, for example, is a great idea, but the character limit of Twitter precludes you from getting much value and descriptions can be truncated.

Another is probably from the CloudSight API. If an image is tweeted twice (such as a retweet), you might get two different descriptions (as this first one demonstrates, and then this second one). On top of this, not all images are very clear and context is hard to convey, as in this one showing wheelchair demonstrators in Seoul.

Regardless, given the current state of accessible images on Twitter, this tool is awesome. As I write this I see more and more people testing Alt Text Bot, so I expect that, even if this is just a proof of concept, more good things will come as a result.

The next image is me being excited about this, along with both descriptions that Alt Text Bot provided.

alt=man in gray shirt smiling thumbs up beside man in black jacke. https://t.co/LhqnvgE2cs – @aardrian

alt=3 men standing near closed white framed glass door. https://t.co/clRfm7BIAG – @nealrs @aardrian