Improbable Icon

Ensuring entity creation

v10-0-0
#1

During my playtests, I have encountered that creating entities may timeout (at least on my local machine). Since I want to ensure that entities are indubitably created I have written a piece of code that should ensure this behaviour, even when commands timeout.

I would like it if others, and especially people from Improbable could check this for potential pitfalls. I have tried to keep all angles covered but without intimate knowledge of the network stack I would love a second opinion here.

First, I reserve an entity id. I do this because a CreateEntity command may fail but still create the entity. By having the entity id beforehand I can verify whether creation did complete or not:

void Initialize() {
            SpatialOS.WorkerCommands.ReserveEntityId(response => {
                if (response.StatusCode != StatusCode.Success) {
                    // this must(!) succeed; do it again!
                    Initialize();
                    return;
                }

                var entityId = response.Response.Value;
                [CALL ENTITY CREATION CODE WITH THE ABOVE VALUE]
            });
}

Secondly, I invoke the CreateEntity command with the reserved ID and when it fails I do an extra check if it really failed or whether the response didn’t arrive properly:

void Create(entityId, prefab, entity) {
            SpatialOS.WorkerCommands.CreateEntity(
                entityId,
                prefab,
                entity,
                response =>
                {
                    if (response.StatusCode.Equals(StatusCode.Failure))
                    {
                        // if the entity doesn't exist then creation failed; otherwise it is created
                        // but the command failed to return a value in time
                        if (SpatialOS.Universe.Get(entityId) == null)
                        {
                            Debug.Log("Failed to create " + prefab + ": " + response.ErrorMessage);
                            Create(entityId, prefab, entity);
                            return;
                        }
                    }

                    Debug.Log(prefab + " " + entityId + "created");
                }
            );
}

The only caveat I can think of currently is that this may go in an infinite loop (and going deeper on the call stack, potentially causing a stack overflow) if entity creation doesn’t happen.

When testing this I did encounter a weird situation, I could get following error when creating an entity:

23:26:19.669 INFO [EngineLogMessageHandler] [Worker: UnityWorker0] Failed to create Moon: EXPIRED: Entity Id reservation for Id<Optional[1807]> could not be found.It might have been expired or never existed. -[WorkerLogger:Unity]

How long do reservations exist? Can they expire within a few seconds?

#2

Hey draconigra, that looks like a good approach - I’ve used a similar pattern too!

Two thoughts:
Checking whether the entity was created by looking in SpatialOS.Universe will only find it if the entity was created and has been checked out by your worker. If the entity is outside your worker radius, doesn’t match your worker’s read ACL, or there’s some delay in delegating to your worker, it could exist but you might not find it in SpatialOS.Universe.
A more reliable approach would be to check its existence with a specific entity id query (HasEntityId in Unity), though this is more expensive.

Secondly, creating entities may fail if your simulation is overloaded. If that’s the case, you risk further overloading (and stack overflow) by instantly retrying as your current code does.
Consider retrying with exponential backoff (like TCP) if you’re doing lots of entity creation.

I’m afraid I don’t know how long reservations are valid for, but that is what EXPIRED refers to.

Karl

#3

That is some seriously good feedback; thank you! I am going to implement the suggested improvements. I don’t think that the entity query’s added load will cause that much of an issue (especially combined with the exponential backoff); I do need to take into account that that query can fail as well and retry it if it errors :slight_smile:

#4

Hi @draconigra, sorry to get back to you so late on this (Karl very thoughtfully pointed out you were still waiting for an answer).

By default, entity id reservation lasts for 5 minutes.