Is the Apple Vision Pro the future of software development?
Updated: Dec 5, 2023
DISCLAIMER: This is my opinion, take it with a grain of salt.
I am a very busy person, most of the time I'm writing code for some software project. And I do this in front of my glorious 3-monitor gaming PC setup at home. I work remotely and constantly traveling, so unfortunately I can't bring my workstation with me.
You could pair your laptop or smartphone to a portable monitor to gain an extra monitor (or in the case of a smartphone, to work more comfortably), but that takes too much space for my taste. Laptops are also big in some cases, specially the gaming ones. Surely there must be a way to have that desktop PC experience for when you have to write code on the go, or do any PC work for that matter, all packaged into a pocketable solution. And I think we may finally be close to having the solution for that, kind of...
Enter Augmented Reality (AR)
In recent years, technology has taken giant leaps forward, transforming the way we interact with the world around us. One such groundbreaking innovation is Augmented Reality (AR). As the boundaries between the physical and digital realms blur, AR has emerged as a fascinating technology with endless possibilities.
Augmented Reality is an interactive technology that superimposes computer-generated images, sounds, or other data onto the user's view of the real world, thereby enhancing their perception of reality. Unlike Virtual Reality (VR), which creates a completely immersive experience, AR supplements the physical environment with digital elements. AR can be experienced through various devices, such as smartphones, tablets, and notably, AR glasses.
Many consumer AR glasses like the Rokid Air Max, Virture One XR and XREAL Air (previously known as Nreal) have been entering the market. And with the upcoming Apple Vision Pro, which promises a great immersive experience (though at a hefty price!), new and better products will start appearing in the market.
All these glasses are intended for consuming media, like watching videos or playing videogames, but can they be used for writing software?
Let's find out.
My AR setup
In my case, after much research I decided to go with the XREAL Air. They are the best looking glasses, the image quality is high and the text is clear. I use it mostly for stereoscopic 3D gaming and with the latest 120hz update, for regular smooth PC usage.
Last week I had to travel to another part of the country, so I decided to experiment and bring my AR glasses with me instead of my laptop. The idea was to access my desktop PC remotely using a remote desktop solution like AnyDesk.
If you were wondering, my full setup consists of the following hardware/software combo:
Hardware:
XREAL AIR glasses
The glasses also have built-in speakers and microphone, so I don't need an additional headset for meetings. They are not perfect but get the job done.
AYA Neo Air
The Neo Air is a fully capable Windows/Linux machine with a gaming handheld form-factor. I'm a PC gamer, so the Air is a huge deal for me because I can bring my PC games wherever I go. It can also be connected to an external USB-C display which is perfect for this use case, even if you don't use a remote desktop solution.
What if you don't care about gaming though?. You can use any of the Xreal supported smartphone devices. If you don't already have one of the supported devices and are looking for a new device, I recommend you picking up a Samsung one, since you can use Samsung DeX for getting that desktop feeling. The only caveat of this is that you won't be able to leverage all the tools available on Windows and Linux, that is if you don't use a desktop remote solution of course.
Bluetooth Keyboard + Mouse
These are a no-brainer, the regular keyboard and mouse are required to have a comfortable programming session.
Software:
NReal Nebula Beta for Windows
Released some time ago, this application allows you to have up to 3 virtual monitors, which you can look at by moving your head thanks to the 3DoF capabilities of the glasses. You can configure the zoom level and the curvature of the screens for more comfortability. At the time of writing, Nebula is on beta on Windows so you need to request access to it on their official Discord channel. Nebula is also available for Android and Mac OS.
AnyDesk Trial
This is only required if you intend to access your PC at home or office, if you are working locally then this isn't needed at all.
So, does it really work?
We finally get to the most important question of this post, does it really work?. For me, the answer is yes, but of course it's not without its flaws. You do need a bit of time to get used to it though. We are still not there yet but at least for me, this is a really good solution that I'll definitely use again the next time I need to travel.
So what are the caveats in my experience?, hopefully these will help you decide whether or not you want to try this:
I can't use the dark theme in Visual Studio Code
Maybe I'm nitpicking, but I found a bit hard to read the text using the dark theme in Visual Studio Code. I had to switch to the light theme. It's not a deal breaker, but still wanted to mention it.
It's uncomfortable for extended periods of time
This was expected as using the glasses for extended periods of time is uncomfortable for me. One thing that I noticed though is that I can focus more on my work since I literally have a screen attached to my face. Having a screen attached to your face makes it harder for you to constantly look at your phone for example, since it's a bit difficult with the glasses on. This also forces me to take regular breaks, so I feel like my productivity increased.
You need a more capable machine for Nebula to work smoothly
Since the idea is to be as pocketable as possible, having a more powerful (and bulkier PC) defeats the purpose of this setup. My experience with 3 screens felt really choppy when programming, since doing any action like moving the mouse cursor, opening menus and general operating system usage is not smooth. The Neo Air just doesn't have enough processing power. It works relatively well if you use Nebula for its intended purpose, which is consuming media. Let's say for example that you are watching 2 videos (a video on each screen) while reading an article on the other screen, it works well. Another problem is that text is a bit blurry when using Nebula (unless you zoom all the way in), so it gets a bit tiring having to read blurry text for long periods of time.
For these reasons I don't use Nebula when developing. This means that I have to work with just one screen, but it's not that big of a deal, at least for the project I'm currently working on.
But in the end I can't really complain, and the future looks bright with all the new and more capable Windows handheld devices like the ROG Ally, which will hopefully make this experience much more smoother.
Here's my two cents. Let me know in the comments what do you think about this, do you also develop software on your AR glasses?
