Improbable Icon

Streaming Video Over Spatial

unity
how-to

#1

If I wanted to stream low resolution video over spatialos, does anyone have an idea what the best way to do that would be?

i was assuming id be setting a user’s webcam to a render texture inside unity, and maybe sending an xy value of the pixel and pixel color only if it changed from the last frame until this one.

I know there are probably other compression codecs i could use, too.

Just wanted to get some thoughts and feedback.

AnTennA


#2

Hi @AnTennA!

My knee-jerk reaction to your post is “you’re probably going to have a bad time”. YouTube has some guidelines about recommended bit-rates for video, and even for 240p video you’re still looking at between 300 and 700 kbps (source). If you’re doing this for each player and you get a lot of players together in the same area, you’ll quickly be exceeding the bandwidths most MMO games try to allow on the player-end. You’ll probably also end up putting a lot of stress on the runtime and might suffer from poor performance if the runtime struggles to keep up with pushing through so much data.

With all those disclaimers about why this may or may not be a bad idea out of the way, I’ll actually try suggesting a solution or two. You could either:

  • Do something similar to the VoIP tutorial, just sending video data instead of audio data

  • Have a state on an entity which is a URL/some kind of reference to a video-streaming server. Instead of piping the data directly through SpatialOS (and risking really bad runtime performance), you could have a state on an entity which is a URL/some kind of reference to a video-streaming server separate from SpatialOS (really naive example: a raw Twitch stream link). You could then have your client worker look at the state on the entity and go to that url, and stream the data directly into the client instead from a third-party source (and benefit from their good compression algorithms).

It might be that neither of these solutions work. Let us know how it goes. I applaud your bravery!


#4

Thanks, Charlie! is there a tool in the console to look at the stress being put on the workers?


#5

If you’re running a cloud deployment (e.g. not via spatial local), you can go to the ‘metrics’ page for your deployment. The metrics pages contain a load of metrics about the resource usage of your deployment. Probably most relevant to you would be the ‘Workers’ dashboard (click on the ‘dashboards’ button in the top right and select the workers dashboard from the drop-down), which contains information about how much data is being sent to and from workers (“Bridge to Worker State Updates”) and what the latency is like for your workers. You can also see Receive/Send queues which may fill up over time if the runtime can’t keep up with the load for a particular worker.