Wordle on Vision Pro

An immersive 3D Wordle game that allows Apple Vision Pro users to place virtual letter cubes in the real world and solve the puzzle in mixed reality.

This project is built on Apple sample codes. See

• Incorporating real-world surroundings in an immersive experience
• Transforming RealityKit entities using gestures

Video Demo

Check out the video demo on YouTube.

Learning Pathway

This project is built step by step in the following order. If you need help with any step, please refer to the complete code for reference.

1. Set up the starting code

• Download the sample code from Incorporating real-world surroundings in an immersive experience. This sample code sets up a basic ARkit project with RealityKit that allows you to place a virtual cube in the real world by tapping.

• Set up the project in Xcode and run it on your Apple Vision Pro device. This will serve as the starting point for the Wordle game. Read the source code and the documentation to understand how the basic framework works.

  • SceneReconstructionExampleApp is the main entry point of the app. It initializes and holds the data model EntityModel, sets up the UI window ContentView and the immersive space CubeMeshInteraction.

    Learn more about RealityKit Entity.

2. Add gesture control

• Download the sample code from Transforming RealityKit entities using gestures. This sample code demonstrates how to use the GestureComponent to allow users manipulate virtual objects with gestures.

• Try running the sample code. Read the source codes as well as the documentation to understand how the GestureComponent works and how to use it.

• Integrate the GestureComponent into your project.

  • Start by copying the entire Package folder to your project root directory and add it to your project. Then you need to call registerComponent and installGestures to set up. Finally, add GestureComponent to each cube entity in the addCube() function.

  Important: The GestureComponent sample code is written in Swift 5, and does not comply with Swift 6 concurrencty checking. See this thread for more details. Try to fix it by yourself or refer to the fixed code in this repo for the solution.

  Learn more about the modular architecture of RealityKit.

• By now, you should be able to tap to place a cube and manipulate it with gestures.

3. Replace primitive cubes with custom meshes

• Download Cube[A-Z].usdz from the Resources folder in this repo. These are the meshes for the letter cubes. Add them to your project. (Model credit: Sketchfab)

• In the addCube() function, replace the primitive cube with the mesh cube. Load the mesh from the file and call generateCollisionShapes() to set up the collision detection.

• Define a LetterComponent as follows and add it to the cube entity. This component will store the corresponding letter of the cube.

  struct LetterComponent: Component {
      let letter: String
  }

• Add a Picker to ContentView to allow users to select a letter from the alphabet. Add a selectedLetter property to EntityModel to store the selected letter.

• Spawn a letter cube with the selected letter when addCube() is called.

• By now, you should be able to pick a letter, place letter cubes in the real world and manipulate them with gestures.

4. Add input bases and handle collisions

• Create an addInputBase() function to add a base to the entities that players will place the letter cubes on. This will be a simple short cylinder mesh. Remeber to generate collision shapes for the base and add necessary components.

• Spawn five input bases in the scene on initialization of RealityView in CubeMeshInteraction.

• Handle collision events so that each input base can keep track of all contacting cubes. This will be useful when we check the answer against the target word.

  • Define a BaseContactComponent as follows and add it to the base entity.

    struct BaseContactComponent: Component {
        var contactingCubes: [Entity] = []
        var mostRecentCube: Entity? {
            return contactingCubes.last
        }
    }

  • Create onCollisonBegan() and onCollisionEnded() functions in EntityModel to handle collision events. Add and remove the cube entity accordingly from the BaseContactComponent.contactingCubes of the input base entity.

  • Subscribe to CollisionEvents.Began and CollisionEvents.Ended in CubeMeshInteraction to detect collisions.

    Learn more about RealityKit Event.

• By now, there should be five floating input bases that can properly collide with the letter cubes and keep track of the contacting ones.

5. Add a target word and check the answer

• Download list of words from GitHub or this repo. There are two lists of words, one for all possible target words and another for all valid words to guess.

• Create a WordPicker struct to handle picking and validating words.

  struct WordPicker {
      /// Returns a random word from the list of target words.
      static func getRandomWord() -> String {
      }
  
      /// Returns if the word is a valid word to guess.
      static func isValidWord(_ word: String) -> Bool {
      }
  }

• Add a targetWord property to EntityModel to store the target word.

• Add a Check Answer button to ContentView. Complete the checkAnswer() function in EntityModel to check if the current word is the target word. Store the references of the five input base entities in a property to access them easily. Update the material color of the input bases to indicate the correctness of the answer.

• By now, you should be able to place letter cubes on the input bases and check visually if the word is correct.

6. Add a history view and complete game logic

• Create a CheckedAnswer struct to store the checked word and the correctness/color of each letter.

  struct CheckedAnswer {
      let word: String
      let colors: [UIColor]
  }

• Add a checkedAnswers: [CheckedAnswer] property to EntityModel to store the history of checked answers.

• Display the history of the checked answers in ContentView with proper colors.

• Disable the Check Answer button when the target word is guessed correctly or there are six checked answers.

• Alert the user when the target word is guessed correctly, the game is over or the input word is invalid.

• Change the Start Game button to Restart Game when the game is over. Carefully reset the game state when the game is restarted.

Resources

• Introductory visionOS samples

• Gaming sample code projects