Multiplayer VR with NetCrafter

Before starting this tutorial you need:

• OpenWorlds VR for Third Person with Game Creator Support v1.3.1 or higher installed (Unity Asset Store)
• NetCrafter imported
• SmartFoxServer 3 running locally
• Java JDK 21+ and Apache Maven installed (same as the Chat/Character tutorials)
• NETCRAFTER_VR_EXAMPLE added to Scripting Define Symbols (Edit → Project Settings → Player → Other Settings)

1. Open the OpenWorldsVR sample scene that ships with the asset (the one containing a working VR character — check the asset's own Documentation/ or Example Scenes/ folder for the exact scene name)
2. Enter Play Mode and confirm:
   • The character moves with the thumbstick
   • Head tracking follows your HMD
   • Both hand controllers track and the avatar's arms follow via IK
3. Stop Play Mode. File → Save As, and save a working copy as:

Assets/Scenes/VRMultiplayerScene.unity

Work in VRMultiplayerScene from now on — do not modify the original sample scene.

Desktop / No-Headset Mode

To allow keyboard WASD control when no headset is connected (or when the headset is removed):

1. In the Hierarchy, find the OpenWorldsVRManager GameObject
2. In the Inspector, enable the PCtoVR checkbox

With PCtoVR enabled:

• WASD keys control movement
• Mouse controls look direction
• Keyboard jump works
• Input switches automatically between VR headset and keyboard based on whether the headset is being worn

Before writing any networking code, it helps to know exactly which transforms and components drive the VR character, since multiplayer sync has to reproduce them on a remote avatar that has no XR rig of its own.

PlayerIKSolver (the CCD-style IK solver) is compiled into OpenWorldsVR.dll with no source shipped in the project. It was reflected directly against the compiled assembly to confirm its real public API — the field/method names below are not guesses. Two relevant facts it exposes:

• remotePlayer (bool) — flips the solver into "read targets from network data" mode
• remoteHeadTarget, remoteLHandTarget, remoteRHandTarget (GameObject) — a first-class remote-player path already built into the SDK: when remotePlayer is true, the solver reads IK targets from these instead of the live XR followers

This is what RemoteVRCharacterController uses: it creates three small driven GameObjects under the remote avatar, adds its own PlayerIKSolver instance in remotePlayer mode pointed at them, and every frame sets their local position/rotation directly from the network stream (no lerp — the solver's own ikBlendTime handles smoothing).

What we sync

Open the NetCrafter window: Tools → Pivec Labs → NetCrafter Pro. Click New Schema and name it VRCharacter.

SmartFox Settings tab

In the NetCrafter window, go to Settings → SmartFox Settings tab and configure:

Project Settings tab

In the NetCrafter window, go to Settings → Project Settings tab and configure:

ClientToServer message — PlayerUpdate

In the NetCrafter window, go to Definition → Messages tab. Click + to create a new message.

• Set Direction to ClientToServer
• Name it: PlayerUpdate
• Set Transport to: ExtensionRequest
• Set Server Handler Name to: HandlePlayerUpdate

Then add the following fields using the + button in the Fields section — 31 fields total, grouped here by what they represent (add them all to the same message):

Root (4):

Head (7):

Left hand (7):

Right hand (7):

Animator + misc (6):

ServerToClient events — leave Server Handler Name blank on all three

PlayerStateUpdate:

In the NetCrafter window, go to Definition → Messages tab. Click + to create a new message.

• Set Direction to ServerToClient
• Name it: PlayerStateUpdate
• Set Transport to: ExtensionEvent
• Leave Server Handler Name blank

Then add the same fields as PlayerUpdate, except colorIndex (that's only sent once, via PlayerJoined — RemoteVRCharacterController never reads a colorIndex field from a state update), plus:

That's still 31 fields total: the 30 PlayerUpdate fields minus colorIndex, plus playerId.

PlayerJoined:

Same pattern — Definition → Messages tab, + to create a new message, Direction ServerToClient, Transport ExtensionEvent, Server Handler Name left blank. Name it: PlayerJoined.

Then add the following fields:

PlayerLeft:

Same pattern — Definition → Messages tab, + to create a new message, Direction ServerToClient, Transport ExtensionEvent, Server Handler Name left blank. Name it: PlayerLeft.

Then add the following field:

Server Actions

In the NetCrafter window, go to Definition → Server Actions tab. Click + to create a new server action.

Configure:

• Handler Name: HandlePlayerUpdate
• Trigger Message: PlayerUpdate
• Response Event: PlayerStateUpdate
• Logic Type: BroadcastAll

1. Code → Build Code → Validate Definition, then Generate All Files
2. Code → Server Code → Generate Server Code
3. Build JAR (mvn package) — wait for BUILD SUCCESS
4. Deploy to SFS3

Key generated files land in Assets/PivecLabs/NetCrafter/Definitions/VRCharacter/Generated/Client/:

• PlayerUpdateMessage.cs
• PlayerJoinedEvent.cs / PlayerLeftEvent.cs
• PlayerStateUpdateEvent.cs
• SmartFoxConnectionManager.cs

Enable the generated type guard

1. Edit → Project Settings → Player → Other Settings → Scripting Define Symbols
2. Add NETCRAFTER_VR_GENERATED
3. Apply

Both NETCRAFTER_VR_EXAMPLE and NETCRAFTER_VR_GENERATED should be present now.

Step 1Duplicate the base model, not a live runtime instance

In OpenWorldsVRManager's Inspector, find List of Player Models and locate the ModelPrefab reference for the character you're using (e.g. LabMan). Drag that prefab asset into the Hierarchy, and rename the instance RemoteVRCharacterPrefab.

Step 2Add network components

On RemoteVRCharacterPrefab, use Add Component to attach:

• RemoteVRCharacterController
• VRColorManager
• PlayerNameDisplay

RemoteVRCharacterController.Awake() adds its own PlayerIKSolver instance and CapsuleCollider at runtime — do not add either manually.

Step 3Set up the name display

Identical to the Character tutorial's B3 step 4:

1. Right-click RemoteVRCharacterPrefab → UI → Canvas, name it NameCanvas, Render Mode → World Space
2. Position X=0, Y=2.2, Z=0, Rotation 0,0,0, Scale 0.01, 0.01, 0.01
3. Rect Transform: Width 200, Height 50
4. Right-click NameCanvas → UI → Text - TextMeshPro, name it NameText, centre-aligned, font size 36
5. On PlayerNameDisplay: drag NameText into Name Text; leave Camera Transform and Billboard Target empty (both assigned/derived at runtime)

Step 4Save as a prefab, then remove from the Hierarchy

Drag RemoteVRCharacterPrefab into:

Assets/PivecLabs/Examples/VR/Prefabs/RemoteVRCharacterPrefab.prefab

Delete it from the Hierarchy afterward.

Components on the final prefab

Create the NetworkManager

1. Hierarchy → Create Empty, name it NetworkManager, reset its Transform
2. Add Component → VRNetworkManager
3. Drag RemoteVRCharacterPrefab.prefab into the Remote Character Prefab field

Add the ConnectionManager prefab

In the NetCrafter window: Scenes → Prefab Generator → Create ConnectionManager Prefab, then Scene Assistant → Add to Existing Scene if it isn't already in your Hierarchy.

Scene Events wiring

Scenes → Scene Events → Add Router to Scene, then wire:

VRNetworkManager subscribes to RoomJoined itself in Start() via NetCrafterEventDispatcher — no manual wiring needed for that one.

VRNetworkManager.cs is the top-level coordinator — same shape as CharacterNetworkManager from the Character tutorial, with VR-specific differences:

• Connect() — called by the login button; calls SmartFoxConnectionManager.ConnectAndLogin(playerName), not Login() directly. SmartFoxConnectionManager no longer opens its socket automatically in Start() — doing so left it sitting idle from scene load until the player finished typing a name, which most SFS3 server configs will drop for exceeding the unauthenticated-connection timeout, forcing an app restart. ConnectAndLogin() opens the socket now and logs in once the CONNECTION event actually fires.
• OnRoomJoined() → starts WaitForLocalPlayerThenSend(), a coroutine that polls until OpenWorldsVRManager.Instance.player exists. This wait is necessary because OpenWorldsVRManager spawns the character prefab asynchronously (LoadPrefabCoroutine) — it is very likely still null the instant the room is joined.
• Once the player exists, it locates HeadFollower/LHandFollower/RHandFollower via FindAnyObjectByType<T>() (they're scene singletons, not children of player — see Part A3) and starts the send loop.
• SendPlayerUpdate() — runs at 20 Hz (50 Hz while the Speed Animator parameter indicates movement). Computes head/hand pose relative to the player root (InverseTransformPoint/rotation delta) and packs everything into a PlayerUpdateMessage.
• OnPlayerJoined() / OnPlayerLeft() — spawn/destroy remote avatar GameObjects.
• OnPlayerStateUpdate() — forwards the deserialized event straight to the matching RemoteVRCharacterController.SetNetworkState(...).

Assign in the Inspector:

VRUIManager.cs manages the same two panels as CharacterUIManager — a login screen and a game panel — with one deliberate omission: it never touches Cursor.lockState/Cursor.visible. VR input comes from the XR controllers via OpenWorldsVRManager, not the desktop mouse, so there is no cursor to lock or unlock. (OpenWorldsVRManager already manages Cursor.lockState itself for its PC-to-VR fallback mode — VRUIManager deliberately stays out of that.)

Create the Canvas

Right-click Hierarchy → UI → Canvas (Screen Space - Overlay for a desktop test build, or a world-space canvas parented under the rig for a pure-VR login flow), name it UICanvas.

Login Panel

1. Right-click UICanvas → UI → Panel — name it LoginPanel
2. TMP Text title: VR Demo
3. TMP InputField PlayerNameInput, placeholder Enter your name...
4. TMP Button ConnectButton, label Connect
5. TMP Text StatusText

Game Panel

1. Right-click UICanvas → UI → Panel — name it GamePanel, set inactive
2. TMP Text PlayerListText

Attach and wire

1. Empty GameObject UIManager, attach VRUIManager
2. Wire all Inspector fields
3. Wire ConnectButton On Click → VRNetworkManager.Connect()

1. Code → Config Generator → Refresh Preview
2. Generate VRCharacterZone.zone.xml
3. Deploy Config to SFS3

Or deploy manually to:

C:\Users\YourName\SmartFoxServer3\server\zones\VRCharacterZone.zone.xml

Restart SFS3 and confirm in the server log:

{VRCharacter}: VRCharacterExtension initialized

Start SmartFoxServer 3

Run SmartFoxServer3.exe. Confirm VRCharacterExtension initialized in the server log before proceeding.

First player

1. Put on your headset, press Play in Unity (or launch a standalone build)
2. Enter a name, click Connect
3. Confirm you can move and your hands/head track normally — the local experience is unchanged from Part A2

Second player

Multiplayer VR testing needs a second tracked client. Options, in order of convenience:

• A second headset on a second PC running a standalone build
• A second headset on the same PC if your OpenXR runtime supports it (uncommon)
• Enable PCtoVR on OpenWorldsVRManager for a desktop-only second instance — this lets you verify networking and IK reproduction without a second headset, though head/hand tracking will just reflect the desktop input mapping rather than real HMD motion

Expected result:

• Both players see each other's avatars
• A remote avatar's head turns and hands move to match the other player's actual tracked motion, reproduced through that avatar's own PlayerIKSolver (in remotePlayer mode)
• The player list updates in both instances; closing one instance removes its avatar from the other

NetCrafter generates the Java server extension skeleton. You must extend it with join/leave/disconnect logic. Replace the generated VRCharacterExtension.java with the listing below, and replace the generated HandlePlayerUpdate.java with the handler listing.

package com.piveclabs.vrcharacter;

import com.smartfoxserver.core.ISFSEvent;
import com.smartfoxserver.core.SFSEventParam;
import com.smartfoxserver.core.SFSEventType;
import com.smartfoxserver.entities.Room;
import com.smartfoxserver.entities.User;
import com.smartfoxserver.entities.data.ISFSObject;
import com.smartfoxserver.entities.data.SFSObject;
import com.smartfoxserver.extensions.SFSExtension;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;

public class VRCharacterExtension extends SFSExtension {

    private final Map<Integer, Room> userRooms = new HashMap<>();

    @Override
    public void init() {
        addRequestHandler("PlayerUpdate", HandlePlayerUpdate.class);
        addEventHandler(SFSEventType.USER_JOIN_ROOM,  this::onUserJoinRoom);
        addEventHandler(SFSEventType.USER_LEAVE_ROOM, this::onUserLeaveRoom);
        addEventHandler(SFSEventType.USER_DISCONNECT, this::onUserDisconnect);
        trace("VRCharacterExtension initialized");
    }

    // ── Join: track room, announce to existing players ──────────────────

    private void onUserJoinRoom(ISFSEvent event) {
        User user = (User) event.getParameter(SFSEventParam.USER);
        Room room = (Room) event.getParameter(SFSEventParam.ROOM);
        if (room == null || user == null) return;
        userRooms.put(user.getId(), room);

        int    colorIndex = user.getId() % 8;
        int    spawnIndex = user.getId() % 8;
        String playerName = user.getName();

        // Tell existing players about the new joiner
        ISFSObject joinMsg = buildPlayerJoined(user.getId(), playerName, colorIndex, spawnIndex);
        Collection<User> others = room.getUserManager().getAllUsers();
        others.removeIf(u -> u.getId() == user.getId());
        if (!others.isEmpty()) send("PlayerJoined", joinMsg, others);

        // Tell the new joiner about everyone already in the room
        for (User existing : room.getUserManager().getAllUsers()) {
            if (existing.getId() == user.getId()) continue;
            int ec = existing.getId() % 8;
            int es = existing.getId() % 8;
            ISFSObject existingMsg = buildPlayerJoined(existing.getId(), existing.getName(), ec, es);
            send("PlayerJoined", existingMsg, user);
        }
    }

    // ── Leave: clean map, announce to remaining players ─────────────────

    private void onUserLeaveRoom(ISFSEvent event) {
        User user = (User) event.getParameter(SFSEventParam.USER);
        Room room = (Room) event.getParameter(SFSEventParam.ROOM);
        if (user == null) return;
        userRooms.remove(user.getId());
        if (room == null) return;
        ISFSObject leftMsg = buildPlayerLeft(user.getId());
        Collection<User> remaining = room.getUserManager().getAllUsers();
        if (!remaining.isEmpty()) send("PlayerLeft", leftMsg, remaining);
    }

    // ── Disconnect: use cached room (getLastJoinedRoom is null here) ─────

    private void onUserDisconnect(ISFSEvent event) {
        User user = (User) event.getParameter(SFSEventParam.USER);
        if (user == null) return;
        Room room = userRooms.remove(user.getId());
        if (room == null) return; // leave already fired and cleaned up
        ISFSObject leftMsg = buildPlayerLeft(user.getId());
        Collection<User> remaining = room.getUserManager().getAllUsers();
        if (!remaining.isEmpty()) send("PlayerLeft", leftMsg, remaining);
    }

    // ── Helpers ──────────────────────────────────────────────────────────

    private ISFSObject buildPlayerJoined(int playerId, String name, int colorIndex, int spawnIndex) {
        ISFSObject obj = SFSObject.newInstance();
        obj.putInt("playerId", playerId);
        obj.putString("playerName", name);
        obj.putInt("colorIndex", colorIndex);
        obj.putInt("spawnIndex", spawnIndex);
        return obj;
    }

    private ISFSObject buildPlayerLeft(int playerId) {
        ISFSObject obj = SFSObject.newInstance();
        obj.putInt("playerId", playerId);
        return obj;
    }
}

package com.piveclabs.vrcharacter.handlers;

import com.smartfoxserver.extensions.BaseClientRequestHandler;
import com.smartfoxserver.bitswarm.data.TransportType;
import com.smartfoxserver.entities.Room;
import com.smartfoxserver.entities.User;
import com.smartfoxserver.entities.data.ISFSObject;
import com.smartfoxserver.entities.data.SFSObject;
import java.util.Collection;

public class HandlePlayerUpdate extends BaseClientRequestHandler {

    @Override
    public void handleClientRequest(User sender, ISFSObject params,
                                    TransportType transport) {
        Room room = sender.getLastJoinedRoom();
        if (room == null) return;

        ISFSObject response = SFSObject.newInstance();
        // Authoritative sender id first — never trust a client-sent playerId.
        response.putInt("playerId", sender.getId());

        response.putFloat("posX", params.getFloat("posX"));
        response.putFloat("posY", params.getFloat("posY"));
        response.putFloat("posZ", params.getFloat("posZ"));
        response.putFloat("rotY", params.getFloat("rotY"));

        response.putFloat("headPosX", params.getFloat("headPosX"));
        response.putFloat("headPosY", params.getFloat("headPosY"));
        response.putFloat("headPosZ", params.getFloat("headPosZ"));
        response.putFloat("headRotX", params.getFloat("headRotX"));
        response.putFloat("headRotY", params.getFloat("headRotY"));
        response.putFloat("headRotZ", params.getFloat("headRotZ"));
        response.putFloat("headRotW", params.getFloat("headRotW"));

        response.putFloat("lHandPosX", params.getFloat("lHandPosX"));
        response.putFloat("lHandPosY", params.getFloat("lHandPosY"));
        response.putFloat("lHandPosZ", params.getFloat("lHandPosZ"));
        response.putFloat("lHandRotX", params.getFloat("lHandRotX"));
        response.putFloat("lHandRotY", params.getFloat("lHandRotY"));
        response.putFloat("lHandRotZ", params.getFloat("lHandRotZ"));
        response.putFloat("lHandRotW", params.getFloat("lHandRotW"));

        response.putFloat("rHandPosX", params.getFloat("rHandPosX"));
        response.putFloat("rHandPosY", params.getFloat("rHandPosY"));
        response.putFloat("rHandPosZ", params.getFloat("rHandPosZ"));
        response.putFloat("rHandRotX", params.getFloat("rHandRotX"));
        response.putFloat("rHandRotY", params.getFloat("rHandRotY"));
        response.putFloat("rHandRotZ", params.getFloat("rHandRotZ"));
        response.putFloat("rHandRotW", params.getFloat("rHandRotW"));

        // animState is int, grounded is float (0f/1f) — match the C# client's
        // wire types exactly (see Part B1's "why grounded is a float" note).
        response.putInt  ("animState",  params.getInt("animState"));
        response.putFloat("grounded",   params.getFloat("grounded"));
        response.putInt  ("stateHash",  params.getInt("stateHash"));
        response.putFloat("stateTime",  params.getFloat("stateTime"));
        response.putInt  ("colorIndex", params.getInt("colorIndex"));
        // Lets each peer disable THIS player's remote head/hand IK instead of
        // resolving toward a placeholder pose — see Part A2's desktop-mode note.
        response.putBool ("isDesktopMode", params.getBool("isDesktopMode"));

        Collection<User> others = room.getUserManager().getAllUsers();
        others.removeIf(u -> u.getId() == sender.getId());
        if (!others.isEmpty())
            send("PlayerStateUpdate", response, others);
    }
}

After editing the Java files, rebuild and redeploy: click Deploy to SFS3 in the NetCrafter window → Code → Server Code tab.

Then restart SmartFoxServer 3 and press Play in Unity to test.

You have a working multiplayer VR game. Here are natural next features: