Everything You See On-Screen Is Not Really There

Each time you–and billions of others around the world–look at a screen, you are seeing things that are not there. There is no video, there is no picture, there is no text. Your screen is made up of pixels. These pixels (named by combining pix- as in picture/pics and -els short for elements) are small collections of a single red, green, and blue light. However, unlike a group of light bulbs, pixels are much smaller–typically filling up a screen on the order of hundreds of pixels per inch, with thousands to a billion pixels per screen.

How does this work, exactly?

Modern computers (from phones to gaming computers to ATMs) have pixels that make up the screen, but what exactly happens “under the hood”?

Each individual pixel is made up of three numbers where each number defines how bright the red, green, and blue lights should be. In modern times, these values range from 0 to 255, but in the future it’s certainly likely that technology advances and allows the computer to go from 0 to 511, or 0 to 1023, allowing for something like 16 to 64 times the amount of colors to be displayed. Deep inside the computer software of the digital display, there is something known as an array of pixels, which is an ordered list of what color combinations of red, green, and blue each pixel should emit.

You may notice I keep talking about red, green, and blue. It’s true, that is all any of these devices can display. But clearly computers and phones and all have more than red green blue. What about yellow, for example?

This is not the color yellow!
See the options where R: 255 G: 255 and B: 0? The color yellow is generated by only turning all the way up red and green pixels (255 is the maximum value) on a screen while keeping blue off.

This is where the magic starts to happen. When you look at a screen, your eyes really only process the red, green, and blue. Your eyes do not actually receive any yellow light, since the screen can only display RGB intensities. But, your brain is able to turn this flood of tightly-packed RGB color into images.

This is why you’ve probably heard the dangers of “blue light” which is known to cause cancer, diabetes, insomnia, sleep disorders, and more. It comes from the blue part of each pixel. This is why looking at almost any screen counts as “blue light”: even though the screen looks white, it’s actually the RGB elements turned up all the way to full brightness. It’s also why night shift works: night shift turns the blue light down to zero for every pixel on your screen.

Animation and Video: Reality Powered By Illusion

The GIF above are not actually moving. Animations, videos, and anything else that looks like it moves is an illusion, caused in a very similar way to how flip-book animations makes a bunch of static, non-moving images appear animated.

Unless you are using a gaming computer or a very high-end laptop, the array of pixels that make up your screen is updated by flashing on-and-off with new RGB intensities around 60 times per second. Gaming monitors and fancy laptops tend to update around 144 times per second. Your eyes, however, cannot update this fast. They can only see a light flicker on-and-off around 15 times per second before it looks like it’s on continuously (even if it is still flickering), which is why some older televisions look like they are flickering. It’s because they are!

Your screen, this grid of pixels, flashes on and off around 60 times per second, every second it is powered on, and yet you can’t see it. To humans, it looks like it is on all the time, full of animation, but this is not actually true at all. There are, of course, health implications of this: zoom fatigue being one of them, but headaches, dry eyes, and strained ocular muscles have been around for decades.

Why Does This Matter?

TikTok, known famously among young Americans for its uncanny and relevant recommendation algorithm, doesn’t actually know what is in the videos.

During work hours I am a software engineering intern at Amazon where I am working on the Amazon iOS app, used by millions of people around the world. Frankly, when you use any app, website, or screen-related activity, there are engineers behind it controlling exactly how bright each of the millions of RGB elements is on your screen. For example, Meta (formerly FaceBook) is hiring a UX Researcher to continue their research in finding out exactly which RGB combination makes users click, swipe, and spend more time on their devices. They’re able to do this because color is critical in consumer decisions. Well funded and well researched, take a look through a library or online for scholarly articles on how color impacts user behavior.

This is not an exaggeration.I have written code in school and in my career that precisely sets how bright the Red, Green, and Blue lights that make up a given pixel are supposed to be, and so have all of the other engineers I have ever worked with who have touched what the user actually sees.

Further, when you scroll through the feed of TikTok’s for you page or Instagram’s explore page, you are given recommendations of videos and images served to you by state of the art machine learning algorithms. These algorithms can’t actually understand what you are looking at. They have no idea what you are actually watching since the video or picture doesn’t actually exist. Remember, the picture or video is a rapidly-flickering grid of red, green, and blue lights–it is an illusion in your brain. Rather, what the computers see is more along the lines of “hey, this pixel is 30% red light intensity, 65% green light intensity, and 100% blue light intensity” (a dark sky-blue to a human).

The algorithms work to determine what combinations of color in certain parts of the video brings you back. Some users like bright green (high green, low red and blue) in some parts, with other colors in other parts. Modern algorithms are able to identify clusters of pixels as objects like dogs or cats or your cousin Dave, but all the algorithm can see is a list of pixels, they can never see like we can. Computers don’t “know” what we see, and I doubt they ever will even if it seems like they can.

Conclusion

When you eventually head out of this article, remember what you are actually looking at. A picture of your family is NOT your family, but rather tiny flickering red, green, and blue lights shining directly into your eyes. All these technologies that “connect” us are really just illusions that sometimes make you feel connected, which in part caused us to go through the loneliest times in human history, and it’s a good idea to banish phones from the bedroom.

Having this intimate understanding of how the technology around me works, I’ve slowly stopped watching tv and using my devices (down to 30 minutes of screen-time on my phone, article coming soon). Instead, I take walks after work to the library, local book stores, through local trails, and just through the neighborhood. It’s also why I’m known to not respond to texts past 9 pm. In the end, it’s impossible for a young software engineer like myself to avoid screens, but having healthy boundaries with technology and understanding that screens are merely an illusion helps me orient myself in the modern world.

Writing articles like this is new to me, I’ve never really done it before but it’s a lot of fun. I’ll keep practicing and getting used to it but there’s a lot of interesting things I want to share about technology, climate change, and maybe even book reviews (I’ve recently been reading nearly 2 books a week). Let me know what you want to see in the comments below, and if you want to get updated when I post, sign up here! I haven’t finished making any sort of email stuff yet, so don’t worry if you don’t get a confirmation. Thank you very much for reading.

One thought on “Everything You See On-Screen Is Not Really There

  1. Hi there, you’re reading a comment. Feel free to leave one (you shouldn’t have to sign up or anything like that)

Leave a Reply

Your email address will not be published.