WebXR Template

This template allows you to create your own WebXR app, that lets multiple users interact with each other on any compatible device they own (e.g HoloLens 2, Oculus Quest 2, Macbook etc) in real-time using Websockets.

Users can also be visualised as 3D Avatars through the integration of Ready Player Me.

This lets you to interact with other users, across devices, with animated and customise-able 3D Avatars.

These avatars are also controllable with Virtual Reality and Mixed Reality headsets, as demonstrated in this GIF.

Summary:

Building on that last point..

PLEASE READ: Using this template involves you using your own AWS account to create cloud resources.

This means that you will be billed by Amazon for usage of those resources.

You are responsible for what you create using this library, it is VERY important that you fully understand what it does before you use it.

If you are uncomfortable with your understanding of what this library does, or you are not familiar with AWS/Terraform/Serverless Framework - then it is my recommendation that you DO NOT use this library.

Deploying resources with this template will result in you getting billed by AWS, you need to understand the nature of - and be comfortable with how - these bills are generated.

What does this template do?

This template helps you create all cloud resources and setup a local host development environment for your own custom multiplayer WebXR application.

Toggle the below tabs to understand how, click here for further reading.

Summary

This template allows you to create a multiplayer WebXR application that can render on different devices, by creating a unique identifier for each device (e.g though user log in), then storing that device’s positional data in a database, each device then uses that positional data to render the other users in WebGL.

The below image demonstrates how this concept works across a Mixed Reality Headset (HoloLens 2), Virtual Reality Headset (Oculus Quest 2) and Laptop (Macbook Pro).

Tech Stack

This template is powered by Terraform, Serverless Framework and Next.js, the breakdown of what they each do is as follows.

Terraform is responsible for creating all cloud resources, excluding lambda functions.

Serverless Framework creates the lambda functions that run the Back End business logic (e.g retrieving DynamoDB data) and deploys them to the API Gateway created by Terraform.

Next.js is the Front End that, once statically served from the S3 bucket – renders the 3D environment, communicates with the Back End via HTTP requests / Web Sockets and authenticates with AWS Cognito via the Amplify library.

App Flow

The way a user would then interact with the app is as follows:

A user visits the url for your web app using their devices web browser, the Next.js pages and content is served to them from the S3 bucket.
Once the page loads, the user signs in using their AWS Cognito details.
Once they have successfully logged in:
- their positional data is sent in real time to the Web Socket API and stored in DynamoDB for other users to render and see
- the WebSocket API also returns the real time positions of all other users and 3D models within the scene that is stored in DynamoDB
- the HTTP API is queried and profile photos are retrieved to be visualised for other users rendered positions

How to use this template

To use this template follow the below steps.

Step 1: Install

Look at the Installation instructions to install the required libraries.

Step 2: Unwrap

Follow the steps outlined in the Setup instructions until you get to Step 3, at that point please bear the below points in mind:

choose the WebXR template by running the command unwrap WebXR.
if you decide that you’d rather create the AWS Secret yourself manually, then the JSON format you want to adapt is below – be sure to replace all values as appropriate to your configuration.

				
					{
    "tf_sls_service_name":"my-wrapperjs-config",
    "tf_sls_next_stage":"dev",
    "tf_sls_next_region":"eu-west-2",
    "tf_sls_next_root_domain_name":"jamesmiller.blog",
    "tf_sls_next_domain_name":"dev.jamesmiller.blog",
    "tf_state_s3_bucket":"my-wrapperjs-config.jamesmiller.blog",
    "tf_state_s3_key":"terraform.tfstate",
    "next_ready_player_me": "jamesmillerblog"
}

Step 3: Gobble

Continue on from Step 3 until you’ve completed Step 6 of the Setup instructions, you can now run gobble dev to run the application and access the webpage on https://localhost:3000.

Please note WebXR API requires https and Wrapperjs uses an autogenerated SSL certificate – if an error message like the below appears, this is expected and you can click to proceed onto next step.

Once you’ve loaded the page, a login form should appear.

To log into your web app, you’ll need to create Amazon Cognito credentials, first step to doing this is to find your Cognito user pool.

Then you need to add a new user, to do this you need to click on the ‘Create user’ button.

Fill out the form like shown below (please note you do not need to enter a phone number), then click ‘Create user’.

Please note, the email address you enter will receive an email with the password you’ve created – the user will need to enter this password to log into the site.

Use the credentials you’ve created in the log in form.

You will be prompted to create a new password on log in, fill this out.

And there you go – you’re in!!

Controls are as follows:

Desktop / Laptop: use your ‘W’ / ‘A’ / ‘S’ / ‘D’ and ‘↑’ / ‘↓‘ / ‘←’ / ‘→’ keys to move
Virtual Reality Headset: walk around within your guardian zone / play area, use controllers to move objects by pointing at and grabbing them.
Mixed Reality Headset: walk around your physical space, use your hands to move objects by pointing at and pinching them.
Augmented Reality enabled mobile device: walk around your physical space.

GO MAKE SOME COOL STUFF 😀

OPTIONAL

You will notice if you check the logs, that there are some error messages.

This is because the WebSocket and HTTP APIs haven’t had lambda functions deployed to them yet.

If you want to get rid of these error logs and enable the real-time multiplayer functionality, then you must:

run gobble serverless deploy to deploy the appropriate backend web socket resources
create AWS Cognito sign in credentials for other users that you want to be able to log in

Table of Contents