Commentary: Facebook’s Accessibility Benefits Everyone

For a little over a year, Facebook has been working on a tool that will describe pictures to people who are blind or visually impaired. It consists of artificial intelligence, and automatically generates captions for the photos we or our friends post. There is no need to install software or take extra steps – the tool is available to anyone using a smartphone, tablet or computer. To get a better sense of how it works, read this Sandy’s View post. I also demonstrate how the technology works in this story from CBS 2 Chicago.

According to this article from CNET, Facebook’s technology promises to get even better! For one thing, photo captions are becoming more enhanced. Now, when I am reading through my newsfeed, some of the picture descriptions also include what people are doing. When this technology was introduced over a year ago, I would only hear something like “image may contain: two people, outdoor, beach, sunglasses.” Now, I might hear something like “image may contain: two people, people smiling, people sitting, outdoor, beach, sunglasses.” In other words, there are more details in the descriptions, and this allows me to better visualize the image. Using the same example, I can picture two people having a good time at the beach!

All people – whether blind or sighted – will also be able to search for specific pictures using these descriptions. So, if I want to find the picture of my friends at the beach, I can type some of the words from the caption in Facebook’s search box. As of the writing of this post, I have not tried out this new feature, but will review it in the near future. I can see this added enhancement helping anyone. Instead of scrolling through our newsfeed or friend’s wall, we can simply search for a particular picture and save ourselves some time that way. This technology currently doesn’t include detailed descriptions, like the color of the clothes someone is wearing, but I am sure it won’t be long before we start seeing them. This would make the new picture search feature much more useful.

Social media has become an important part of everyone’s lives, and thanks to accessibility efforts like those implemented by Facebook, people with vision loss can also be included. For almost a year, I have been using Facebook’s photo description feature, and it has given me a better picture (no pun intended!) of what my friends and family are sharing. Although this technology is still in its early stages, it has certainly come a long way and made a difference for people with vision loss. Better yet, it also includes features, like the picture search, that will one day be useful even to those with sight. I sure am excited to see what it has in store for us in the future!


Commentary: Finding Different Ways to See Pictures

Making photos on social media accessible to users with vision loss appears to be – at least for the time being – the latest trend. Last week, Facebook released its long anticipated tool that automatically generates captions for all of its photos. The tool, which I have previously reviewed, is only available on iOS devices, and there is no need to download additional software or take extra steps to use it. Similarly, Twitter launched a new feature, whereby users can write descriptions of the photos they post. Since I have already written about Facebook’s tool, this commentary is primarily focused on Twitter’s feature, and the pros and cons of both.

Contrary to Facebook’s tool, the new feature in Twitter is not turned on by default. Users interested in captioning the photos they post must first enable a setting on their smartphone. From then on, they will be able to add descriptions to all photos they post in the future, and those of us who are blind can listen to the descriptions with our screen-reading technology. This, I think, will allow people to provide more accurate and detailed picture descriptions. Unlike Facebook’s tool, Twitter’s feature works on both Android and iOS mobile devices.

Twitter’s effort to make pictures accessible to everyone is definitely a step in the right direction. Unfortunately, not everyone (myself included) takes the time to write picture descriptions for all their photos, making them inaccessible to people who can’t see. This brings me to what I think is the biggest drawback to Twitter’s new feature. I wonder how many people will actually turn it on? What’s even more important, how many of them will take the time to write out a description for each photo? I highly doubt that the average person will know about this feature, let alone take a few extra minutes to describe the pictures.

Neither Facebook’s nor Twitter’s features are perfect. While Facebook’s tool provides automatic descriptions, these are still very vague in detail. In contrast, Twitter is giving users control and allowing them to write more accurate and specific descriptions. Rather than seeing these tools as a form of competition, I see both as great alternatives for making pictures accessible to me and countless others. Both methods are still in their early stages, so honestly it is too soon to tell which one works better.

My hat goes off to Facebook and Twitter for striving to make their sites more accessible for people with vision loss. I strongly hope that other social media websites will also take similar initiatives and develop new methods so we all can “see” the pictures our family and friends post. By introducing these new methods, Facebook and Twitter are helping us all get the full picture of what we should and can do to make social media accessible to everyone.

A Step Closer to Accessible Pictures!

Facebook A.I.

A few months ago, I wrote a commentary about Facebook’s initiative to make pictures accessible to people who are blind or visually impaired. I finally got my hands on this tool last weekend, and so far I have been quite amazed. While far from being perfect, I think it’s a great step in the right direction that will benefit blind and visually impaired Facebook users in the near future.


What Is It?


Simply put, Facebook’s new accessibility tool consists of artificial intelligence technology. It is not a software or stand alone device, but rather a feature that will eventually be incorporated into everyone’s news feed. In other words, both blind and sighted people are able to access this feature, and no special steps are needed to make it work. Each picture will be automatically captioned with a description developed by artificial intelligence technology. Blind and visually impaired users can hear the description on their smartphone or computer in the same way they access other text on Facebook.


How Does It Work?


Users log on to Facebook and access their news feed the same way they’ve done it in the past. If they come across a picture while scrolling through the news feed, the user hears something like “[person’s name] added a new photo.” If text is included with the picture, the phone or computer will first read it. Next, it will say something like “photo, this image may contain: one person, smiling, indoors.” Note that the captions will always begin by saying “this image may contain.” My guess is that this is Facebook’s disclaimer letting users know that the descriptions are not necessarily 100 percent accurate.


Other Observations


As mentioned above, this tool tells users how many people it thinks appear in a picture. I found out it is fairly accurate last weekend when I posted a group picture. It told me that there were 16 of us and that we were all smiling! If the picture is from the outdoors, the description may also include other information. A few days ago I came across such a photo, and I was told that it “may contain outdoors, plant, tree, cloud, sky.” When presented with pictures of written text or food, the tool lets users know this information, but it does not give specific details about what kind of food or what the text reads. I also noticed that some pictures are not captioned, but I’m sure this is a technical flaw Facebook will eventually fix.


Suggested Improvements


Like anyone else, those of us who are blind or visually impaired want to know every detail that can be seen in photos. It would be helpful to have more specific descriptions of people, such as what they’re wearing, what’s around them and what they appear to be doing. Images containing food should be more detailed, and – whenever possible – the caption should include what the food is. Many people post pictures with text, often containing important or funny messages. I would love to see a day when this tool could also read the text so that blind and visually impaired users can enjoy the message as well.


Final Thoughts


Kudos to Facebook for striving to make their site more accessible to people with disabilities. This is only the initial version of the tool, so I did not expect it to be perfect or 100 percent accurate. Still, I am extremely amazed with the whole concept. Although the captions are very vague, it truly makes a difference for someone who can’t see to at the very least have some details of the images. I am excited to see what the future holds regarding social media accessibility for people who are blind or visually impaired. Technology has always been a game changer for those of us with disabilities. Now it will allow people with visual impairments to get the full picture and make social networking more enjoyable and accessible.

Commentary: Getting the Full Picture on Social Media When You Can’t See

The modern concept of social media is relatively young and constantly evolving. Websites like Friendster and MySpace began emerging in the early 2000s. Today, websites like Facebook, Twitter, Instagram and LinkedIn are among the most popular social media platforms. The vast majority of content found in these sites is picture and video based. This can present problems for blind social media enthusiasts like myself, who constantly miss out and cannot get the full picture – literally!

Matt King is Facebook’s first blind engineer, and he hopes to give other blind individuals the ability to “see” the pictures on their news feed. He is currently working with Facebook’s accessibility team on an artificial intelligence based tool that will describe the photos people share, and they hope to release it by the end of the year.

As someone who is blind, I often don’t even bother to read – or listen – to the comments people leave about the photos my friends and family share. The reality is that even if I do take a few minutes to read through the comments or captions, I am still missing out on the content. Although my talking computer and cell phone can read text just fine, they cannot decipher what’s in the pictures for me. It’s not that my family and friends purposely exclude me, but the fact is that most of us don’t think to add in a descriptive caption of each picture. I myself am guilty of doing this. Why bother adding a description to my pictures? After all, about 90 percent of my friends on Facebook are sighted.

Now more than ever, pictures play an important role in social media. I guess we could say that a picture is really worth a thousand words in today’s technologically driven world. As an avid Facebook user, I see this all the time. If I’m lucky, my posts that only include text might get seven or eight likes. Meanwhile, the posts with pictures usually get at least 15 or 20 likes. Society seems to be more engaged with visual content, and those of us who can’t see should also be included as much as possible in this trend.

I applaud Facebook’s commitment to accessibility for people with disabilities. Technology has opened numerous possibilities for us, and staying connected with our loved ones and meeting new people has probably become one of the most valuable and cherished aspects. I hope that other social media sites will follow in Facebook’s footsteps, and that they too will strive to make all content accessible to disabled individuals. Better yet, I sincerely hope that one day technology will make it possible for me and other blind individuals to truly get the full picture of the photos we can’t see. Maybe then I will be able to confidently say that I am fully connected with my loved ones and the entire world.