Individuals with visual hindrances utilize social media like every other person, often with the help of screen reader software. However, that technology falls short when it experiences memes, which do exclude alternate text, or alt text, to describe what’s delineated in the picture.
To counter this, analysts at Carnegie Mellon University have built up a strategy to automatically distinguish memes and apply prewritten templates to include descriptive alt text, making them clear through existing assistive technologies.
Memes will be pictures that are copied and afterward overlaid with slight varieties of text. They are frequently comical and pass on a shared encounter, however “if you’re blind, you miss that part of the conversation,” said Cole Gleason, a Ph.D. student in CMU’s Human-Computer Interaction Institute (HCII.)
“Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention,” said Jeff Bigham, associate professor in the HCII. “Many people use memes, and so they should be made accessible.”
Memes to a great extent live within social media platforms that have obstructions to including alt text. Twitter, for instance, enables individuals to add alt text to their pictures, yet that feature isn’t in every case simple to discover. Of 9 million tweets the CMU specialists analyzed, one million included pictures and, of those, simply 0.1 percent included alt text.
Gleason said essential PC vision techniques make it conceivable to describe the pictures fundamental every meme, regardless of whether it be a celebrity, a crying infant, a cartoon character or a scene, for example, a bus upended in a sinkhole. Optical character acknowledgment procedures are utilized to decipher the overlaid text, which can change with every iteration of the meme. For every meme type, it’s just important to make one template describing the picture, and the overlaid text can be included for every iteration of that meme.
Be that as it may, working out what the meme is expected to pass on demonstrated troublesome.
“It depended on the meme if the humor translated. Some of the visuals are more nuanced,” Gleason said. “And sometimes it’s explicit and you can just describe it.” For example, the complete alt text for the so-called “success kid” meme states “Toddler clenching fist in front of the smug face. Overlaid text on top: Was a bad boy all year. Overlaid text on bottom: Still got awesome presents from Santa.”
The group likewise made a platform to make a translate of memes into sound as opposed to text. Clients search through a sound library and drag and drop elements into a template. This system was made to translate existing memes and pass on the conclusion through music and sound effects.
“One of the reasons we tried the audio memes was because we thought alt text would kill the joke, but people still preferred the text because they’re so used to it,” Gleason said.
Sending technology will be a challenge. Regardless of whether it was incorporated into a meme generator website, that alt text wouldn’t be automatically copied when the picture was shared via social media.
“We’d have to convince Twitter to add a new feature,” Gleason said. It could be something added to an individual cell phone, however, he noticed that would put the weight on the client. CMU scientists are right now working on related projects, including a browser extension for Twitter that endeavors to include alt text for each picture and could incorporate a meme system. Another project looks to incorporate alt text into the metadata of pictures that would remain with the picture any place it was posted.
This work was introduced not long ago at the ACCESS conference in Pittsburgh. Different analysts associated with the research incorporate HCII postdoctoral individual Amy Pavel, CMU undergraduate Xingyu Liu, HCII assistant professor Patrick Carrington, and Lydia Chilton of Columbia University.
Disclaimer: The views, suggestions, and opinions expressed here are the sole responsibility of the experts. No News Feed Central journalist was involved in the writing and production of this article.
