HoloLens Case
We had a request from a client to implement an integration demonstration for HoloLens with an existing network infrastructure database. The goal of our application is to provide assistance to the system administrator in monitoring and maintaining the IT inventory in an organization.
Our Expectations from HoloLens
The system administrator puts on the HoloLens computer and starts walking through the office. Our HoloLens application has to use the built in camera from HoloLens to recognize the computers in our database. Once it identifies a computer – our application should create a hologram hanging over the computer that will display the information it has found about the computer in our database – operating system, network to which it is connected, owner, etc…
The application should also determine if there are any known issues with the device or if it is performing any unexpected activity. For any discrepancies with the baseline that are found the app should highlight the affected computers with color.
Each hologram should also have an interface so the user can perform functions such as: disconnect from the network, mark the device for further inspection or retrieve additional information on it.
The Realities of HL
Since the built in HoloLens camera has a rather low resolution and shapes of computers are very common to each other, it is very difficult to recognize even the model of computer, not to mention distinguishing between two different computers of same model.
Therefore we decided to use completely different images – like cans, triangles, circles, which had to be put on tables first. Another possible alternative is to use stickers or QR codes on or near the devices.
With a bit of a workaround we were able to overcome these difficulties in the end however. After our application identified an image it used some predefined metadata, that related to the image and used it to decide which information to show in the icon above that image.
So the final solution was such:
- The user places images over different tables and enters image recognition mode.
- Our application identifies the images, creates a persistent hologram above them and stores their locations so that after the application reboot icons will be loaded at exactly same places.
- Remove the images from tables and put on their places real devices instead – laptops, mobile phones…
As a result of these actions the icons were hanging over correct device in the correct places and the app was able to display the correct data.
HoloLens Technologies
Unity is the recommended platform for the development of applications for HoloLens so we used it too.
For the image recognition we used the Vuforia library – https://vuforia.com/. It allows the creation of a cloud database of images and includes the metadata for them. While our application is in recognition mode it uses Vuforia to scan input video from the camera and identify the images.
Once an image is identified – Vuforia provides us with the metadata which we have already assigned to that image in the cloud, and using that metadata we can show the appropriate icon above any found image.
Basic Data about HoloLens
HoloLens is a wireless headset with see-through holographic lenses with transparent monitors for each eye placed high in the field of view.
It also has several cameras and infrared sensors. Tech specs can be found here microsoft.com/store/d/microsoft-hololens-development-edition/
HoloLens can work without recharging for 3 hours.
Scanned room is stored as a wireframe or, as it is called by Microsoft – mesh.
Before using it the user should perform the following operations:
- Connect to WiFi (or just have some WiFi reachable)
- Enter room scanning mode – HoloLens will ask the user to walk though the room to make a map of it and store it.
- HoloLens is also able to determine which room it is by the available/connected WiFi networks.
Now we can start using our application.
The application can place holograms in two different ways:
1. Relative to user.
Let’s say that the user just started application. The users head is used as the center of coordinates (0, 0, 0). So if we decided to place a hologram in the coordinates (0, 1, 1) user will see it in one meter in front of him on height equal to user’s height minus plus one meter.
If the user stops the application and then moves to another place in the room and launches the application again our app will create hologram with same coordinates – the hologram will appear in a different place than before because center of coordinates was moved.
2. Anchor it to some surface or position in room (referred to as spatial mapping).
In this case is our application first created hologram in some location in the room and tells that it should be anchored exactly to that place – then no matter where the user moves to and when the application is launched next time – hologram will remain in same place.
In our application we gained experience with both ways –first to always show menu above user, –second to place holograms above images.
Conclusions from HoloLens usage experience
Although we believe Hololens is a big step forward in the march towards augmented/mixed reality, it looks like it’s not yet perfect for our case. We found a few weak spots like:
- Narrow field of view – holograms doesn’t cover all user view – just almost 60% of square in front
- Can only correctly work indoors.
- Tracking of position is not perfect – as a result the image is jumping slightly when a user moves.
- Very sensitive to environment changes – better not to change the placement of objects in room once it was scanned, it also doesn’t like it when somebody walks in front of the field of view.
- Needs good lighting in the room.
Anyway even at this moment HoloLens is a great way to attract attention to future possibilities – we see it as a great and promising tool for many different types of implementations, both practical and entertaining.
