Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multiplayer docs, a little org and fix vroid link #105

Merged
merged 1 commit into from
Jul 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/create/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ In addition, Webaverse supports traditional 3D models. Any content creation plat

**I want to package my creations from...**

- [VRoid Studio](./import-vroid-studio.md)
- [VRoid Studio](./import-vroid.md)
- [Blender](./import-blender.md)
- [MagicaVoxel](./import-magicavoxel.md)
12 changes: 0 additions & 12 deletions docs/engineering/graves-notes/graves-notes-intro.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,6 @@ const {offset, duration} = audioSpec;

# Systems/Algorithms

## CRDT theory

Conflict-free Replicated Data types are the technology that webas multiplayer system is based on. Necessary knowledge for anyone who might work on multiplayer

- Massive overview of CRDTs from creation to optimization. https://bartoszsypytkowski.com/the-state-of-a-state-based-crdts/


## L-system Theory

L-systems are used to effiently store and generate fractals like plants and crystal structures. This will be an effective way to generate plants for the game world.
Expand Down
82 changes: 82 additions & 0 deletions docs/engineering/multiplayer/multiplayer-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
id: multiplayer-overview
title: Multiplayer Overview
---

Some notes on the current state of multiplayer development, as well as some quick what-you-need-to-know to get started with developing for multiplayer

## What can I do in multiplayer?
- Jump seamlessly between single and multiplayer
- Switch my avatar, either from an avatar in the world or from the menus
- Speak with my voice
- Switch my voice pack and voice engine
- Have my character's mouth respond to my voice with mouth movement
- Change my name, and see other user's names as a nameplate over their head
- Chat, and have my character speak whatever I chat
- Drop items into the world and use them
- Drop pets and vehicles into the world and activate them
- Pick up weapons and damage mobs
- Die and respawn at current world spawn point

## Important Files
The important, high-touch documents for multiplayer are:
- avatars/avatars.js
- app-manager.js
- character-controller.js
- chat-manager.js
- players-manager.js
- webaverse.js
- universe.js
- world.js

## Where do I start?
The main entrypoint to the Webaverse app is the Webaverse class in webaverse.js -- this is where everything gets set up and the main loop starts. The world, local and remote players are updated from here.

Multiplayer starts from enterWorld in **universe.js**

## Key Concepts

#### CRDTs
Webaverse is a peer-authoratative system. Objects in the world can be moved and manipulated by peers, and these changes will be reflected by the server out to other peers. It is simple in theory, but in practice this can potentially lead to conflicts where peers don't agree on where something is, or *what* it is.

**Conflict-free replicated data types** or CRDTs are a set of data structures which can be merged in such a way that any inconsistencies are eventually resolved, and all peers will have the same document state. Additional reading can be found here:
[Conflict-free_replicated_data_type](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type)

#### Websockets, WebRTC and WSRTC
Previously, Webaverse used a server-authoritative WebRTC implementation for voice and data. However, this has significant scalability issues and was generally not fun to work with. The current implementation uses pure WebSockets. In the future we will adopt WebTransport and turn on AV1 and WebCodecs once they are well-supported in Safari and Firefox.

## Multiplayer-Related Packages

#### zjs
zjs is a purpose-built API-compatible clone of yjs for multiplayer realtime games. Most of the concepts and documentation for yjs applies, so please read before attempting to work on the multiplayer.

The TL;DR is that zjs is a CRDT-backed document store. When changes to the Doc object occur, zjs will handle any conflicts and then send change events with the final state at that moment to all connected clients.

Any object in Webaverse which has a bound state -- players, app managers and tracked apps -- will be subscribed to these change events. This is done using the "observer" pattern, and zjs maps and arrays are observable objects.

#### wsrtc
Actual socket communication is handled by wsrtc, another webaverse project, which moves app data and voice packets over websockets. wsrtc is heavily bound to zjs. Messages received from peers are passed into the server's own zjs instance, where conflicts are handled and final state is passed back out to peers.

An important note about wsrtc is that this will eventually be replaced with redis or a similar distribute key-value store and possibly phased out entirely.

#### totum / metaversefile
Totum (formerly metaversefile, and still called this everywhere) is an API for Webaverse to load composable apps and for said apps to communicate with the Webaverse core. The metaversefileApi contains useful helper functions for initializing and interacting with metaverse apps, or for getting important core data into those apps. A good example would be accessing the local player's app manager, or checking if the scene is loaded.

## Development
#### Player Controllers
The character-controller.js file holds the LocalPlayer, RemotePlayer and the Player base class. Everything related to character state, including transform, movement, actions, wearing, etc is here. NPCs are also a type of player

The actual visual avatar display is largely handled by the Avatars class in the avatars/avatars.js file. The character controller sets the velocity of it's avatar class instance, which then handles the animation update.

Player objects are held in the PlayersManager class in players-manager.js, this is initialized when the client connects to a room. The PlayersManager object is a zjs map of player objects, which are also zjs maps, containing component information that might change. Players sync most things via "actions", but also have information like name, instance ID and their chosen voice engine and voice pack.

Player objects have a playerId, and are stored in the playersArray of the PlayersManager class instance

The instanceId of the avatar can be found inside the player map's avatar key, or through calling CharacterController.getAvatarState()

#### Apps
The AppManager class (app-manager.js) handles setting up, tearing down and updating apps. Apps can be avatars, wearables, buildings, anything that lives in the world with the player. All AppManagers are bound to state, which can be local or networked. world.connectState sets up all of the AppManager bindings locally, while world.connectRoom sets up the bindings globally.

When an app is loaded, it is bound to a map or array in zjs. Each app has an instanceId, which is a 5-digit alphanumeric hash. The instanceId of an object on one client should match the next.

The instanceId is a useful key for getting the app. Each app has a list of components, a contentId and an instanceId (which should be unique).
40 changes: 40 additions & 0 deletions docs/engineering/multiplayer/mutliplayer-stack-flow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
id: multiplayer-stack-flow
title: Multiplayer Stack Flow
---

This is the stack-flow of initialization calls and event responses. This is subject to change and should be updated to UML.

### App.jsx Initialization
> universe.js > enterWorld # user pressed multiplayer button or went to multiplayer room
>> world.js > connectRoom()
>>> players-manager.js > playersManager.bindState # start listening for players added and removed to this array
>>>> players-manager.js > bindState.observe(**observePlayersFn**)

>> world.js > connectRoom()
>>> character-controller.js > localPlayer.bindState # Start listening for changes to this player
>>>> character-controller.js > bindState.observe(**observeAvatarFn**) # Listen for changes to the avatar
>>>>> character-controller.js > localPlayer.attachState
>>>>>> app-manager.js > appManager.bindState # Listen to changes to individual apps
>>>>>>> app-manager.js > bindState.observe(**observeAppsFn**)

### Observer Event Handlers
player **observeAvatarFn**
> character-controller.js > **observeAvatarFn** > syncAvatar()
>> character-controller.js > **observeAvatarFn** > transplantApp()

appManager **observeAppsFn**
> (respond to added and removed events on apps)

app-manager.js **observeTrackedAppFn**
> (if event contains transform update, update app transform)

### Remote Player

players-manager.js **observePlayersFn**
> (remote player joins) > new RemoterPlayer()
>> character-controller.js > remotePlayer.bindState
>>> character-controller.js > bindState.observe(**observeAvatarFn**)
>>>> character-controller.js > remotePlayer.attachState
>>>>> app-manager.js > appManager.bindState
>>>>>> app-manager.js > bindState.observe(**observeAppsFn**)
2 changes: 1 addition & 1 deletion docs/engineering/wsrtc/wsrtc-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ title: WSRTC Quick Start

## Introduction

WSRTC (WebSockets RealTime Communication) is part of a multi-room system that uses binary websocket messages to transfer y.js state across the network. Combined with the World and Universe systems, that's how multiplayer data is updated within the App.
WSRTC (WebSockets RealTime Communication) is part of a multi-room system that uses binary websocket messages to transfer zjs state across the network. Combined with the World and Universe systems, that's how multiplayer data is updated within the App.

Wsrtc also uses web codecs to perform voice encoding and decoding and as well as some spectrum analysis for doing things like animating the mouth flaps of the avatar/rig.

Expand Down
107 changes: 107 additions & 0 deletions docs/engineering/zjs/zjs-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---
id: zjs-quickstart
title: ZJS Quickstart
---

# Overview
zjs is a purpose-built API-compatible clone of yjs for multiplayer realtime games. Most of the concepts and documentation for yjs applies, so please read before attempting to work on the multiplayer.

The TL;DR is that zjs is a CRDT-backed document store. When changes to the Doc object occur, zjs will handle any conflicts and then send change events with the final state at that moment to all connected clients.

Any object in Webaverse which has a bound state -- players, app managers and tracked apps -- will be subscribed to these change events. This is done using the "observer" pattern, and zjs maps and arrays are observable objects.

## Where Do I Start for Development?
If you are going to work on zjs you should start by reading through the tests. For any work you do, please write new tests and verify that they work.

#### CRDTs
Webaverse is a peer-authoratative system. Objects in the world can be moved and manipulated by peers, and these changes will be reflected by the server out to other peers. It is simple in theory, but in practice this can potentially lead to conflicts where peers don't agree on where something is, or *what* it is.

**Conflict-free replicated data types** or CRDTs are a set of data structures which can be merged in such a way that any inconsistencies are eventually resolved, and all peers will have the same document state. Additional reading can be found here:
[Conflict-free_replicated_data_type](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type)

#### zjs document hierarchy

Z.Doc
- AppManager.appsArray -- all apps in the world
- PlayersManager.playersArray -- a list of all playerMap objects
-- CharacterController.playerMap -- all the data synced about the player, including playerId, apps and transform
-- CharacterController.avatarmap (getAvatarState) -- A map of avatar data

Here is an example of an app stored as a Z.Map
```json
{
"instanceId": "rum6g", // shared by all users
"contentId": "https://webaverse.github.io/silsword/",
"transform": [ -6, 0.5, 12, 0, 0, 0, 1, 1, 1, 1, 0 ],
"components": "[]"
}
```

Here is an example of a player stored as a Z.Map -- the player has an avatar and is holding a sword
```js
const examplePlayer = {
"playerId": "GoK3g",
"avatar": { "instanceId": "qgboft" },
"transform": {
"0": -1.3072199821472168, // position x
"1": 1.2576431035995483, // y
"2": 6.1084885597229, // z
"3": 0, // quaternion x
"4": 0.10555386543273926, // y
"5": 0, // z
"6": 0.9944136142730713, // w
"7": 16.66699981689453 // time delta
},
"actions": {
"e": [
{
"type": "wear",
"instanceId": "nxAup", // reference to the silsword user is holding
"actionId": "Hgos0"
}
],
"i": [ "bafb82" ]
},
"apps": {
"e": [
{ // avatar
"instanceId": "qgboft",
"contentId": "./avatars/scillia_drophunter_v15_vian.vrm",
"transform": {
"0": 0, // position x
"1": 0, // y
"2": 0, // z
"3": 0, // quaternion x
"4": 0, // y
"5": 0, // z
"6": 1, // w
"7": 1, // scale x
"8": 1, // scale y
"9": 1, // scale z
"10": 0 // time delta
},
"components": "[]"
},
{
"instanceId": "nxAup",
"contentId": "https://webaverse.github.io/silsword/",
"transform": {
"0": -0.9813549518585205, // position x
"1": 1.536096215248108, // y
"2": 6.172163963317871, // z
"3": 0.9454419016838074, // quaternion x
"4": -0.30134329199790955, // y
"5": -0.11808272451162338, // z
"6": 0.037259675562381744, // w
"7": 1, // scale x
"8": 1, // scale y
"9": 1, // scale z
"10": 0 // time delta
},
"components": "[]"
}
],
"i": [ "952bde", "808f75" ]
}
}
```
Loading