Experiential Design Task 3: Creative Experience Design Project Prototype

Zhang Wan Ru / 0368214

10/06/2025—06/07/2025 (Week 8—Week 11)

Experiential Design/Bachelor of Design in Creative Media/Taylor's University

---

Instructions

---

Task

Introduction

This prototype design is to help us find some problems or limitations that users may encounter when using an emotional confession platform like "TreeHoleHearing." Through this process, we can see more clearly which functions are really useful and which areas need improvement. At the same time, it also encourages us to think creatively about solutions, such as allowing them to feel relieved from stress, such as adding the sound of wind blowing leaves to slow down and think about problems. How to make users feel more at ease to express themselves.

Progression

I looked at the assignment requirements and started breaking down my ideas into

Input: Scanned printed pet photos

Output: Tree with buttons and music sounds

Application: Unity 2021.3 + Vuforia Engine

I started by choosing to design a tree image to insert into the Vuforia Developer Portal. This tree was created and redesigned from a free image found on Xiaohongshu.

Fig. 1.1 Tree

The image of my original tree in the Target Manager had the wrong colors, so I designed another tree.

Fig. 1.2 Tree

Then I added the buttons needed for my app according to Mr. Razif’s original teaching process, adding an image as the background and text.

Fig. 1.3 Cavans UI Button

Then I added the words “You have worked hard today.” Placed in the Text Mesh Pro text input.

Fig. 1.4 Text Mesh Pro text input

Next, I searched for audio about wind blowing leaves from CapCut, downloaded it, and converted it to MP3. Then I created a VoicePlayer and clicked Add Component → Audio Source in the Inspector. Then I moved the MP3 into AudioClip, selected my voice audio, checked Play On Awake, unchecked Loop, and then adjusted the volume to 0.7 to make the wind sound gentler.

Fig. 1.5 VoicePlayer

Then I started to add the timing control of playing the voice, and when using the photo connection of Vuforia's recognition target, the audio is played only when the recognition is successful. However, I also set the code, but it seems that it is not successful when I check it. I also checked the problem one by one. To overcome the problem.

Fig. 1.6 Visual Studio Code

I originally placed it in World Space, but due to a debugging error later on, not all UI buttons were showing in play mode. So I switched back to Screen Space Overlay.

Fig. 1.7 Canvas Render Mode

Final Outcome:

Fig. 1.8 Final Outcome

Challenges

• World Space Canvas display anomaly: The message card is very small in the real game, and it is not visible or blocked at first. Later, the render mode of canvas is adjusted.
• Message card mirror inversion: Text (TMP) is displayed in reverse at some angles. You need to adjust Scale. x = -1, or use the LookAt script to keep facing the camera.
• When placing the sound, I placed it as a video, which resulted in a black screen. Finally, I readjusted it to create empty, then added the Audio Source in Add Component and placed it as a sound, and adjusted the volume to 0.7.

What’s Working Now

• The message cards and text in the Canvas are displayed normally; the text size is appropriate, no longer mirrored or too small
• The voice is played after scanning
• When tracking is lost, all elements can be hidden together
• The scene structure is clear, and the UI has a clear division of labor in the space

What’s Next

• Add a 3D model of the tree
• Add an interface for writing messages, allowing users to customize the content
• Exporting the project

---

Reflection 

Experience:

In the process of making the MVP, I ran through the entire process from image recognition to loading 3D models. In short, I used Vuforia to recognize an image, and then after successful recognition, I displayed a 3D model of a tree and a UI where you can leave messages, forming a "message tree hole" scene.

During this process, I learned how to set Vuforia's ImageTarget and also understood the difference between UI display in different rendering modes (such as Screen Space and World Space). I also fixed the model's material mapping problem for the first time and tried to adjust the interaction between Canvas, AudioSource, and Prefab components, which gave me a more intuitive understanding of Unity's overall workflow.

Observations:

When importing models, the most common problem is that the material is missing. For example, the tree model I downloaded from the Asset Store is gray after importing, and sometimes even purple blocks—because the texture is not automatically bound, I need to manually drag the texture to the material ball to display it normally.

In addition, there are many pitfalls in the UI. If the canvas of the message card is scaled too small or the Z axis is too far back, it cannot be seen on the device at all. You have to constantly adjust its position and rendering order to make it "stick" to the correct position steadily.

It is also easy to step on thunder when building, especially if the script has not been compiled or your computer has not installed the support module for the corresponding platform (such as iOS Build Support). Unity will directly report an error and will not export for you.

In addition, the sound part is also easy to be overlooked. When I added the sound effect, I forgot to add AudioSource, and there was no sound at all. Or if I accidentally added two Audio Listeners to the scene, Unity will directly warn you. So now I will check whether these basic components have been added.

Findings:

I found that the material is really critical. Although the model itself is imported, if the texture is not properly bound, the overall visual effect will be much worse. Especially in AR, the texture directly affects the sense of reality. So now I will habitually check whether the material in the Mesh Renderer is set correctly.

There is also UI. The Canvas in AR is completely different from the UI we usually use to make web pages or apps. The UI of World Space is actually more like a 3D object. Its position, angle, and scale must be adjusted like a sculpture, rather than simply dragging a panel out and using it. But I don’t know why it didn’t show up in the end, so I switched back to Screen Space—Overlay.

Although my scene is an MVP and doesn’t look big, I found it necessary to make the functional layers clear. To prevent confusion when adding features or altering the structure later, for instance, it is important to determine which layer the user interface (UI), model, and sound should be placed on.