Improbable Icon

VR in SpatialOS

bestpractice
vr

#1

Hi everyone,

A lot of people have been asking about how to integrate VR and SpatialOS, so I thought I’d write up a short introduction. This won’t deal with the details of how to get VR working in unity and things like that, but should give you an idea of how to do VR in a multiplayer context.

##So, you have a Spatial and you want to VR?

A good place to start from would be a component like this:

component VRPlayer {
    id = .. ;
    EntityPosition base_position = 1;
    Offset head = 2;
    Offset left_controller = 3;
    Offset right_controller = 4;
}

The idea here is that you have a single entity that represents the player’s position along with the location of the head and the the controllers. I’ll come to what the Offset type actually is in a minute, but it represents a position offset from base_position, plus a rotation. You could think of the base position as the point in the game world that the centre of the players room would be at. The head and the controllers are free to move around the base position without changing it.

So why have all three objects in one component? They could arguably be separate entities after all, without needing to worry about offsets and things. Well, they could be, yes, but the general rule is things in SpatialOS should only be entities if they can exist on their own. It’s unlikely that you want to be able to have one hand in one place and the other one 2km away, so it seems to make sense to keep them as a single entity.


###An aside on Efficiency

There are also efficiency gains from storing the positions as offsets from a base position. A basic way of defining the Offset type would like this:

type Offset {
    Vector3f position = 1;
    Quaternion rotation = 2;
}

Where a quaternion might be stored as four floats.
This will be perfectly usable, but is also quite heavyweight! Every time a player moves, there will be a large amount of data synchronized: a vector is three 32 bit floats, and a quaternion might be four 32 bit floats, so that would mean synchronizing 3*(3+4)*32 = 672 bits for each player every time they move.

We can do much better than this. We certainly don’t need the full precision that floats provide. We can also take advantage of the fact that the hands can only move at most a couple of meters from the base position, and the tracking only has limited precision anyway. This means we could get away with using small integers to represent, say, the distance in mm from the base position. Since these will mostly be small, and the wire format SpatialOS uses (protobuf) will represent small ints in much less that 32 bits, we can accurately represent position in much less than 96 bits. Similarly, you can compress quaternions and do far better than four floats.

This should be enough to bring you down to a decent bandwidth, but if you really want to push things, then you might want to consider delta compression. I’m not going to go into details here, but the idea is that a hand is usually pretty close to where it just was, so you don’t always have to send the whole position. Sending small deltas from the previous position can drastically reduce the amount of data you need to sync. But this is a big topic for another time.

For the rest of this post, I’ll be ignoring efficiency and using the uncompressed offset for simplicity, but it’s easy enough to swap it out for a better version.


Sending the data

Once you have your schema defined, you can send the state of your player using something like this:
(again, I’m ignoring efficiency from now on)

class VRPlayerReporter : MonoBehaviour
{
    [Require] private VRPlayer.Writer VrPlayer;

    public Transform Head;
    public Transform LeftController;
    public Transform RightController;

    private void Update()
    {
        var basePosition = transform.position;
        var headOffset = GetOffset(Head);
        var leftOffset = GetOffset(LeftController);
        var rightOffset = GetOffset(RightController);

        VrPlayer.Send(new VRPlayer.Update()
            .SetBasePosition(transform.position.RemapUnityVectorToGlobalCoordinates())
            .SetHead(headOffset)
            .SetLeftController(leftOffset)
            .SetRightController(rightOffset));
    }

    private Offset GetOffset(Transform Child)
    {
        var positoinOffset = Child.position - transform.position;
        var rotation = Child.rotation;
        return new Offset(positoinOffset.ToNativeVector3f(), rotation.ToNativeQuaternion());
    }
}

This code simply reads the position of the three game objects representing the head and controllers, and sets the offsets from the base game object.

The corresponding visualizer looks very similar:


class VRPlayerVisualizer : MonoBehaviour
{
    [Require]
    private VRPlayer.Reader VrPlayer;

    public Transform Head;
    public Transform LeftController;
    public Transform RightController;

    public void Update()
    {
        transform.position = VrPlayer.Data.basePosition.RemapGlobalToUnityVector();

        ApplyOffset(VrPlayer.Data.head, Head);
        ApplyOffset(VrPlayer.Data.leftController, LeftController);
        ApplyOffset(VrPlayer.Data.rightController, RightController);
    }

    private void ApplyOffset(Offset Offset, Transform Child)
    {
        Child.position = transform.position + Offset.position.ToUnityVector();
        Child.rotation = Offset.rotation.ToUnityQuaternion();
    }
}

First and Third person views

Great! But where do these scripts actually go? Unlike most entities, which will look the same on each clients and can therefore use the same prefab, a player in first person had better not be in first person on another player’s machine! We need two different prefabs to make this work, a first person player for ‘your’ player, and a third person player for everyone else. the tricky part is that the same entity ends up looking very different depending on whether that entity ‘is’; the player or not.

The way this currently works in SpatialOS is by using something called asset contexts. You can tell SpatialOS to use a different prefab on the client if that client is authoritative on any of the entity’s components.

You can read more about this here.

What you should end up with is one prefab, VRPlayer@Player, that contains the vr integration, the headtracking, the first person camera etc, along with the VRPlayerReporter script from above, and also another prefab, VRPlayer, that contains no cameras or vr integration, but does have the VRPlayerReporter script.


Wrapping up

There are many things I haven’t covered, such as how to deal with player input and how to actually spawn the player entity, but they mostly aren’t specific to VR. Hopefully this will be enough to get you started with VR in SpatialOS and to turn this:

Into this:

If you have any more specific questions, feel free to post them here.

Good luck!


Getting started with Improbable for VR
#2

@chris is my favourite !


#3

A post was split to a new topic: Unity throws errors when building for Linux and VR Support is turned on in the Player Settings


#4

Hi Chris,

Can you provide further details on setting up the asset contexts as the documentation points to the blank project but doesn’t have anything that’s mentioned in the doc’s and is unclear on how to properly use this.

Also is this setup per entity as we would mostly just want to use this for the player and wouldn’t want to have to create two prefabs for everything.


#5

Thanks for pointing that out, looks like the docs are slightly out of date there.

The way to enable asset context is to add the following to your spatialos.UnityClient.worker.json file

"asset_context": {
    "use_player_context_if_authoritative": true
}

under the bridge section.

This will make unity try to spawn the version of the prefab with @Player appended whenever the client has authority on that entity.
So for example, if your third person player is using the MyAmazingPlayer prefab, the first person one should be named MyAmazingPlayer@Player.


#6

Thanks Chris,

How would you then go about setting different entity templates for each of these contexts given that we currently have a single entity template script defining components and giving permissions? I’m not sure how this would translate to using the different prefabs?


#7

Hey there, sorry for the late reply, I missed your question. Hopefully it’s still helpful.

There are really only two contexts - one context for entities that you have authority over and one for everything else. Usually, everything gets created with the default context, but by using the flag above, you can get it to load prefabs with “@player” appended whenever you have authority on an entity.

For example, if you have two players with prefabs PlayerType1 and PlayerType2, and two clients ClientOne and ClientTwo then on client one, you’ll see PlayerType1@player and PlayerType2, whereas on client two you’ll see PlayerType1 and PlayerType2@player.


#9

Is this method still a valid way of accomplishing this? In the docs it says that the feature is Deprecated and use is strongly discouraged.


#10

We’re still working on the replacement ; the deprecation you see is so that we can role out the new methodology safely, without impacting large ongoing projects :slight_smile:


#11

Hi,

Is there any update on the replacement for the asset context method?

I’ve just discovered SpatialOS and currently have a game architecture that needs something like this. I have a lot of Unity components attached to the first person representation of the player. The third person representation is an entirely different prefab currently.


#12

I wrote this for using a different prefab for VR and non-VR players.
It should cover your usecase too I think!
https://forums.improbable.io/t/running-a-mixed-vr-and-non-vr-game-with-spatialos/2350?source_topic_id=1451

Let me know if you have any questions, also jump into the community Discord and say hi :smile:


#13

Great stuff, thanks Kaffo.

I’ll come and say hi in Discord.