Semester 2: Theme: Environments
WEEK 1: Draft Project Plan
Image Generated by Bing Image Generator – Prompt = a device that improves wellbeing that has a hardware-software interface
This week, we were introduced to the main task for this semester – to design and complete a project from start to finish. We were told that the project should ideally support the idea we have or might have for our final project in semester 3. We were encouraged to think about what skills we might need to realise the project in semester 3 and use this semester to develop these skills.
In semester 1, I developed the idea of an ‘intimacy generator’, a device that collects live data from a user and uses the data to control music or images that create a sense of connection between the user and the device.
I have started to extend this idea into something that perhaps also contributes to the positive wellbeing of the user and hope that this will help me incorporate some of my mental health experience.
Below is a first draft of a project proposal that might meet these aims.
Project
- To design a prototype device that uses real-time data from biosensors to identify a stressed or anxious state, and then to create or play music that is soothing or relaxing until the participant feels calmer
Introduction and Background
- Music is a ubiquitous feature of modern life and the algorithms in our streaming software can generate playlists for our moods
- My final project seeks to use dynamic music created or played in response to real-time biosensor data from an individual that respond to their affect in real time.
- Before I can do this, I need to understand the theory that investigates the relationship between sound and feelings/mood
- I also need to identify suitable technologies that might help me realise my final project
- As music and mental health are large and complex topics, this project will limit its focus on music that is designed to relax or soothe and a focus on anxious mental states
Aims and Objectives
- To use a cross-disciplinary framework to research the existing theory on the links between sound/music and mood/feeling in illness, health, and wellness
- To critique existing approaches to similar projects
- To build a prototype that generates music or plays pre-recorded music in response to live data from a biosensor like a galvanic skin response sensor or a heart rate monitor.
- To use the learning from the above for the semester three final project
Rationale
- Anxiety and stress are common everyday occurrences
- This technology has the potential to offer an individual relief that is guided by their body and is relatively low tech and cheap
Planned Literature Review Questions
- Music therapy and ideas around biofeedback in modulating anxiety or stress
- Anthropology: cultural variation in sound
- Music used to create a mood: music in films and environments
- Physics of sound waves and instruments
- Technology of biosensors
- Review of similar projects or approaches
Draft Project Gant Chart
SWOT Analysis
Strengths
- I have deep knowledge about anxiety, stress and ways that I can be measured in a medical context
- I also have expertise in neurology, physiology, psychotherapy and positive psychology
- I also have skills in music and music production
Weaknesses
- My coding skills have significant gaps and I have limited knowledge of hardware and its strengths and limitations
- There will be a significant time pressure during this semester
Opportunities
- The project may give me opportunities to collaborate with others and to learn about adjacent or related projects or fields
Threats
- There may not be enough time for me to do deep research or develop a good enough prototype
- The department may be limited in what hardware it can offer
Learning Journeys and Readings
Journal Article: “Flesh and metal, re-configuring the mindbody in virtual environments
Journal Article: “The relationship between heart rate and preference for tempo of music”
WEEK 2: Considering Design Strategies
Image: Mood board for Project Idea
This week’s lecture was about design and prototyping. I am familiar with the design cycle in the context of art and graphic design and am familiar with the idea of cycles of iteration. I am less familiar with UI or UX and after the lecture went to the library to find resources on these topics.
I found resources on designing for interaction and have also been looking for research on hardware – software interfaces and ways that existing technologies integrate user data to customise their outputs.
I have also been thinking about the complications of algorithmically driven technology and have been looking for theory that gives context on the wider implications of ‘new media’ and ways that art and design have contributed to the subversion of capitalism through cultural jamming.
Learning Journeys and Readings
Book: “The Little Know It All” – Silja Bilz
Book: “New Media Cultures” – David Marshall
Book – “Culture Jamming – Marilyn Delaure
Group Work -Global Smart Lab Application
Journal Article-Advanced Internet of Things for Personalised Healthcare Systems a Survey
WEEK 3: Interactive Narratives
Image: Title: DOA (2023) – One of my paintings about masculinity, risk and desire.
This week we had a lecture on interactive narratives by Dr Elin Festoy. She describes herself as a Creative Producer and talked about her game My Child Lebensborn. She made a strong case for games as a way to explore challenging or difficult topics. She explained that the interactive nature of games – when planned right – could create novel situations for empathy and that this explains the success of their game.
She said you should start with the “why” of the game and then think about the potential impact of your game on it’s audience and whether they will be interested in the idea that you are presenting.
They talked about using traditional narrative approaches, like the Hero’s journey, to structure the flow through the game. She also emphasised the fact that games allow for users to make their own meaning / journey through the decisions that they make and this is another big benefit of using games instead of a linear story.
I was inspired by the lecture – by the idea of games as a method to create new thinking about difficult topics – and wondered if I could use a game format to explore a difficult topic from my own life that might have resonances with a wider group of people.
I reflected on a difficult experience I had on a train with 2 men who were drunk and wondered about using the situation as the seed situation for a game about masculinity and risk.
I decided to change my project to a game and rewrote my project plan to fit with this.
Project Track: Spatial Storytelling
- I will use my experience on a train with some drunk and intimidating men to make a game about men, masculinity and safety in public spaces.
- The game will be an experiment in trying to start a conversation about a challenging or difficult subject.
- It will draw on current conversations about the ‘crisis in masculinity’ and gender theory as applied to masculinity.
Introduction and Background
- In our lectures we have discussed how games can be a way for people to experience or think about difficult or unusual situations
- It can be a way to create empathy or new ways of thinking about situations as they invite players to step into another person’s situation
- I had a moderately stressful situation on a train provoked by a couple of drunk men and want to use this experience to design a game
- The game will invite users to experience a version of this scenario and hopefully to start a conversation about these kinds of experiences
Aims and Objectives
By the end of this project I will have
- Have conducted research on game theory, and more specifically on gaming and masculinity
- I will research a range of game making strategies and choose the best for the project
- I will design and make a prototype game which may also be the basis for my final project in semester 3
- I will gain user feedback on how the effectiveness of the game
3 Key Technologies
- A desktop or mobile game: Advantage – common ways to play games, already existing game engines and platforms that can be used. Disadvantages – Learning curve likely to be steep. May be tricky to do everything well.
- A VR or AR scenario based simulator: Advantage – if done well could be a great way to bring the story to life. Disadvantages – suspect the learning curve would be even more extreme. This might be a project that I could do in semester 3.
- A table top role playing game: Not sure this is the right kind of ‘technology’ for this semester but it might be a great way to explore the issue and develop and test the narrative aspect of the game. We have had some good experiences of table top role playing games on the course so far – people seem to enjoy doing this. It is low tech and easy to get set up.
3 Key Media Formats
- Text or images: I need to find the best way of communicating the story to the user and suspect this will be combination of text and images.
- Digital interfaces: I think the type of interface will add limits to the affordances of the game. I will need to work through this as I develop more skills.
- Music: The music will also be key in contributing to the mood of the game. I hope to be able to make some music for the game too.
5 Use Cases
- For the casual gamer interested in an interesting / unusual game format and story
- In an education setting – maybe with teenagers – as a way to start conversations
- As part of an art installation asking people to think about masculinity / men / safety / danger
- As an iterative piece of research that can be shaped and reshaped according to conversations
- As a way to support adults to have conversations about safety in the context of a wellbeing group
Rationale
- I have been in many situations where drunk (mainly) men threaten or enact violence towards others and want to use the specific experience on the train to bring attention to the issue
- I think it is an important discussion point for wider society and hope by the end of this project, I will better understand how games might be able to be part of the solution to the problem
Draft Project Gant Chart
SWOT Analysis
Strengths
- I have a longstanding interest in masculinity studies and have researched the topic as part of my undergraduate art degree
- I also have expertise in psychology and mental ill health
- I also have skills in music production, art and design and writing
- I have experience in running wellbeing groups for gay, bi and trans men
Weaknesses
- My coding skills have significant gaps and I have limited knowledge of hardware and its strengths and limitations
- There will be a significant time pressure during this semester
- The project may end up being too complex as I will need to create a range of assets as well as learn how to use a game engine
Opportunities
- The project may give me opportunities to collaborate with others and to learn about adjacent or related projects or fields
Threats
- The game deals with sensitive topics and could also be construed as being part of an ‘identity politics’ agenda
- It may actually stop men talking about the topic
Learning Journeys and Readings
Book Chapter – “Empathy” – in “How to do Things with Videogames”
Book Chapter – “Art” – in “How to do Things with Videogames”
Book – “Explorations in Global Media Ethics” – Muhammad Ayish
Podcast – BBC – “Wages for Housework Then and Now
Book – “The Best Interface is no Interface” – Golden Krishna
Case Study -YouTube Video Game Walkthrough – “What Remains of Edith Finch”
WEEK 4: Reviewing Our Project Plans and Conducting Research
Image: my initial mind map ideation about masculinity and gaming
This week, we had 1:1 conversations with Julian about our project ideas and he gave us ideas for additional reading and tips.
My Notes:
- There is a risk that the game will make people feel defensive and this might be a barrier to it being played or shared.
- Need to identify my KPIs for this semester so that I can measure my progress.
- Could consider use case in schools or learning environments: make it more educational.
- Could the game be considered to be curative or preventative? What is it trying to achieve for / with its audience.
- Would the game fit with the idea of tactical media?
Things to look up
- contextual engineering
- books:
- neuromancer,
- ecological masculinities
- artists and artist collectives:
- blast theory
- PUI collective
- Zack Blas
Mood Board Inspirations (all images taken by me)
Mood Board Explanation
To find inspiration for the project’s aesthetic and theme I trawled through my image bank of things about the theme or that fit with my idea of how the project could look. As mentioned before, I have been interesting in masculinity, safety and desire for a long time and so already had some ideas about how this project might be realised.
Left to right:
- Paintings in the top background: my paintings about mens bodies, desire and risk. They were called the shame series and were painted in 2023. The third of the images is called ‘bums on seats’ and it is set inside a train carriage. It riffs of an idea bout same sex desire in public spaces, and how between men this is a very risky thing to engage in.
- Also in the top row are 3 images of outdoor environments. One from London, one from the Dorset coast and one of a night scene. I want to use images of normal people as a reference for the scenes characters.
- The final small photograph is from an exhibition at God House Tower in Southampton in 2023. The piece was called Queer Bar Light Room by Alec Stevens. Although I didn’t really like the content of the exhibition (felt it was too tame) I did enjoy the aesthetic and I wonder if I could do something similar (neon light type effect).
- Bottom left: more photos of people, environments and ‘vibes’ that I want to consider for the game.
- In the middle is the work of the artist Belkon Ayón whose work I have seen twice. This image was from the recent exhibition in Oxford.
- The image next to it is of a piece by Chris Ofili in the Tate staircase. I like the dreamy quality of the brushwork and am drawn to making digital paintings with loose, textured feels.
- There’s also a Francis Bacon painting in there too (bottom right)
Learning Journeys and Readings
Book -“Masculinities and Mental Health in Young Men
Preliminary Research on Masculinity (done in Miro): https://miro.com/app/board/uXjVLgVUEQg=/?share_link_id=785088795851
WEEK 5: Immersive Design
Image Source: https://unity.com/use-cases
This week we had a day of thinking about immersive design. The morning session was a lecture about the challenges of creating immersive environments using virtual reality, including the limitations of the technology. We thought about it from the perspective of User Interfaces and User Experiences.
In the afternoon, we had an introductory workshop on using Unity to create an interactive box that a user can open in a VR setting.
The session was difficult as the technology did not work on any of our group’s computers. This highlighted another complication of using VR—the interfaces are not commonly used, and there are many hardware interface issues.
I wondered if I should consider using VR for my game – but decided that it would be too complicated and depend on technology and software that I not familiar with. The software would not run on my computer and the university comptures are not always available.
Learning Journeys and Readings
Journal Article-“3D User Interface Design
Online Learning-Designing for Virtual Reality LinkedIn Learning
Book-“The Art of Game Design – A Book Of Lenses
Game Design -Choosing Game Making Platforms
Book – Handmade Pixels -Independent Video Games and the Quest for Authenticity – Jesper Juul
Book- How Games Move Us – Emotion by Design – Katherine Ibister
WEEK 6: Spatial Storytelling
Image: our group work using the Zoe plugin
This session started with a lecture about storytelling and VR and the limitations of VR environments. It was argued that VR worlds need to have stories that bring the aim of the creator to life. They cannot rely on the technology to do this on its own. In the afternoon, we were working on creating an environment using the Zoe platform that we experienced using Meta Quest headsets.
We had time to experience a VR game called Notes on Blindness. We enjoyed the aesthetic of the game, but the game did not feel as immersive as it could have because of the background noise in the class. Therefore, we felt we didn’t get a true experience of what the game intended to communicate.
We later worked as groups to develop a concept for a VR experience. We developed an idea about experiencing New Year’s Eve events from around the world. The user would be able to switch between different cultures celebrating New Years Eve in different ways.
As per other VR sessions, we spent a lot of time trying to get used to the VR environment, working out how to manipulate and place objects. I realised that design in VR has its own issues and that even a small idea takes a lot of time to construct and test.
I continue to look for the best platform to design my game on and this week have also joined a group to apply for one of the WSA Global Smart Lab projects. I continue my research on the intersection between masculinity and gaming.
Learning Journeys and Readings
Group Work – Applying for WSA Global Smart Lab Project
Watched – “Why VR Does Not Promote Empathy”
Podcast – The Artifical Human Can AI Level Up Video Games?
WEEK 7: Wearables Hackathon: Exploring Microcontrollers and Soft Circuit Interactions
There is no reflection on the session as I was absent for this session.
Reviewing the Project Plan and Progress
There is a lot of work to do this term and I spent some time mapping out the time left until submission, excluding other commitments, there are about 14 full days left before the end of term. After this, I want to be using the time to complete the write-ups for this and the other modules.
Mapping out the time, I have split the remainder of my project work into 2 phases. Phase 2 (phase 1 was the research phase) is testing different mini-ideas and game-making platforms. This is to help test out my story writing and design skills and to work out which platform I might be able to use for my bigger prototype which will be phase 3 (see learning journey below).
As time is tight – I plan for phase 2 to be quick and rough and aim to start phase 3 soon. Phase 3 will hopefully include some more developed storyboarding, consideration of aesthetics and some peer testing.
Learning Journeys and Readings
Research – YouTube Video: Comparing Different Game Engines
Game Design – Working on the Storyboard for Danger on the Train
Research – Choosing a 2D Game Platform
Book – “everyone is an entrepreneur nobody is safe” Silvio Lorusso
WEEK 8: AI Prompts
Image: Generated by stablediffusionweb.com: prompt: naked merman surrounded by 5 not-crows and cats in the water at night.
This week‘s session
I really enjoyed this session learning about the limits of large language and image models. Learning about the latent space and how to ‘break’ the process by asking the models to produce things which do not exist or which are circumvent it’s restrictions.
I found it very interesting that it will not produce nudity unless it is tied with specific phrases (like mermaids). I decided to try the same for mermen and it produced a lot of strange images. It refused to show an image of a naked man but did show me the image above of a merman- even though it doesn’t actually look like a merman.
The model that I used did not cope with the term “non-white” and seemed to always generate odd choices for the crotch areas. These men were supposed to be naked centaurs and ‘non-white’. I’m not sure if the image is technically full-frontal nudity or not?
Image: Generated by stablediffusionweb.com – who are these men??
We also did some exercises on prompt writing in pairs and realized how difficult it is to describe an image in words or to generate an image using another person’s text descriptions.
I think having a better understanding of the limitations of the models is reassuring as it shows how much richer human creativity can be. AI is more efficient at producing novelty, but at present, it’s novelty is often stereotyped (e.g. all images are ‘beautiful’ by default), or bizarre. I’m also interested in why certain images are considered inappropriate (like full frontal nudity) and how other people have tried to get around these restrictions.
Learning Journeys and Readings
AI Experimentation – Asking Chatgpt About Nudity and Censorship in AI Image Generation
Game Design – Learning About Scenes in Godot
Book – “Who Can Afford To Be Critical?”
WEEK 9: Spatial Sound
Image: Working at the Mac
This week‘s session
- At the Tuesday session we had an introduction to using Ableton 12 Live to create a 3D sound map which we could then test using the sensors and headphones in the room
- We started out by using 4 zoo sounds
- We were shown how to use the ‘map’ setting with LFOs in Ableton to add a dynamic dimension to the sounds were were placing and I enjoyed experimenting with using it with moving and expanding sounds
- I attended the additional Friday workshop and just before the session recorded some singing on my phone and used this with a recent recording of me talking about masculinity to create a new sound environment
- The aim of the piece was for the person walking around to hear the voice talking about masculinity and then experience varying amounts of overlapping song as they went around
- My attempt was messy and the instructor helped correct my reverb (making sure it applied to whole track rather the individual tracks) and this reduced the muddiness of all the tracks playing together
- Here is a sample of the tracks playing together
- With more time I would have liked to have refined the sounds, picked parts of the song clips that worked better with each other and have remodelled the audio space a bit more
- It has given me an interesting taste of Spatial Audio which I hope to use in the development of the audio for my game project
Learning Journeys and Readings
Learning More About 2D Game Making
Research – Playing Games with an 8-Year-Old
Group Work – Global Smart Labs – Talking About Death Game
Book Chapter – “The Greatest Story Ever Told Final Fantasy IIV”
Easter Break
Image: Museum Boujmans Depot Gallery – from a trip to Rotterdam
One to One Session with Julian 15.04.2025
- We talked about the progression of my idea and planned narrative structure and game play. The game is most like an experiential narrative or a kind of simulation.
- Julian suggested looking up
- Games: Super Columbine, Braid
- The work of Henry Jenkins on participatory media, particularly around ideas about games and their potential to contribute to violence => the game could be about creating empathy through UX and User Response feedback
- The relationship between the military and games and how this is theorised to continue to influence what games look and play like
- Also suggested that I think about this prototype as a teaser / trailer for something more expansive in semester 3
- Talked about testing while collecting stress data from the EEG device and this might give interesting insights about the effect of the game
- Semester 3 could look at making a game for teenagers in a school setting – there may be expertise in the university that I could draw on
- We talked about another idea for the game as a branching narrative that focuses on the impact of life events on the likelihood that someone will become violent later
- We talked about race and identity politics and how challenging it can be unpick tricky topics in public
Plan
For me to work on ideas about audience, characters, UX, and user response for the prototype and to think about semester 3 planning, and if needed, to bring this back to Julian for more discussion
Learning Journeys and Readings
Learning How to Use Godot Game Engine
Working on Danger on the Train Game
Workshop: Global Smart Lab Project Workshop – The Fundamentals of Game Design
Learning How to Make a Space Rocks 2D Game with Godot
WEEK 10: Visual Coding for Interactive Design
Image: Still image from VVVV example “Rings”
This week we had a fascinating introduction to VVVV and its nodal based system. We experimented with examples and learnt how to use these example node processes to build our own ideas.
I attended Kevin’s evening artist talk, where I learnt about the journey he has taken to becoming an artist practitioner. I was impressed with the way that he has used his broad experience in different fields to build his artist practice.
VVVV looks like software that I would enjoy using and if the opportunity arises. I will look for opportunities to learn more about it in semester 3.
Still images of class experiments using VVVV
Learning Journeys and Readings
Artist Talk – Kevin Blackistone
Podcast – BBC The Artificial Human “Can AI Save Darth Vader?
Reading – Vox Article: Which AIs Do Which Tasks Best, Explained.
WEEK 11: Learning to Code with AI
Image: Point and click ‘game’ coded by Claude AI
Lecturer: Mar Canet Sola
Mar gave a positive lecture about the potential for AI large language models to support us to code. In particular he talked about which AI models are currently best for which kinds of coding problems. He encouraged us to think of AI as augmenting and not replacing our process. He said that no matter how good the model is, and even with a very specific prompt, the output still needs 10-20% additional input from a person.
Interesting to think about errors in AI output as either ‘code level’ or ‘functional’ and that function errors can be thought of as useful glitches.
The games that AI made with my prompt:
I asked Claude AI and Gemini to make a game about the potential future where AI takes away jobs from humans. The first result was a sort of fun game where the player has to keep the balance between AI jobs and human jobs:
I asked it to use the style of Google Studios and I think it did a good job.
Then I asked AI to code a point and click style game with an image in the style of Edvard Munch’s scream of a future where AI has taken over and is contributing to extreme beauty standards, pollution and a disillusioned society. The first version was in P5.js and the second made using HTML, CSS and Javascript:
The final task I gave it was to use an image from the computer’s webcam to make an image in the same style so that the user is included in the work. It got as far as taking the image and putting it in the background, but could not seem to also apply the style.
Final Reflections
I am aware that I am suspicious of generally integrating AI into my day to day tasks, mainly because of the large hidden energy cost and the lack of transparency about what these models are going to be used for in the future. None of the models are saying that they are looking at improving the day to day conditions for ordinary people and so far, ‘efficiencies’ usually mean employing less people.
I left the class and Mar’s artist talk with the plan to try out AI in the final stages of my game making process. A few of the results can be seen in my learning logs below.
Learning Journeys and Readings
Game Making – Using AI to Generate the Text
Game Making – Developing the Storyboard and Artwork
Game Making – Creating the Sounds
Final Week
Game Showreel
My final reflection on the semester’s work
Evaluating My Project Against The Brief
This term we have been lucky to have been taught by a very broad range of creative technology practitioners. Each one has inspired me in different ways and it has been very useful to get a sense of the breadth of what a career in creative technologies could look like.
About 3 weeks into the semester, I set myself the aims of investigating game theory and how it might intersect with masculinity, researching different desktop game making strategies and making and testing a game prototype. This was under the project track of ‘spatial storytelling’.
Despite the significant time pressures, competing demands and the fact that I have never made a game before – I am pleased that I managed to achieve all of these aims. Spending time reading the theory has helped me get a sense of some of the debates about the role of gaming in the lives of men and also showed me that there is a definite gap in masculinity studies when it comes to thinking about the role of gaming in the lives of men.
The game was completed as a prototype, a kind of proof of concept and to develop it further I would be looking to thicken the narrative, introduce branching choice elements and work to unify the game aesthetic. The game is currently a mix of new material and repurposed images and sounds from previous works, and I suspect that the difference between the works may confuse some players.
In saying this, this project has helped me realise that ‘remixing’ is an important part of my practice. The themes of my work have remained relatively consistent, and this is why it felt easy for me to pull in old work that considers masculinity, pleasure and risk into the context of the game. Working with autobiography is a way for me to ground my practice and to resist making work that feels inauthentic. Working with pleasure and pornography is becoming a way for me to explore the hidden intimacies that many people enjoy but are embarrassed to talk about. Doing this project, I also realised that I like my work to have a sense of play or fun too. I think that where possible, there should be a permeable membrane between so-called serious and fun topics.
I only had time to conduct limited playtesting. With a few more weeks, I would have specifically liked to ask people what parts of the story they felt worked, where they felt the message became confused, and how they made sense of the dream sequences where the imagery and audio become explicitly sexual. I think this final line of questioning has the potential to become a rich conversation as it asks people to consider the interaction between public personas of masculinities with their private sense of sexualities. As discussed above, these are ongoing preoccupations of mine and I suspect that I will look for ways to ask these questions next term regardless of the medium that I end up working in.
What Did I Learn About My Process
I am still looking for the ‘right’ way to be a creative and to find the digital technologies that I can use for the next stage of my creative practice. I am surprised at how little I have been interested in using my medical knowledge in the course so far but suspect that this is probably a function of detoxing from the career and suspect that psychiatry will start to creep into my work next term.
Making the game has given me the confidence to not be afraid of coding-based projects. I am not sure if I will ever feel like I can code but making games this term has given me a very useful grounding in the way that coding can be used to meld creative ideas with computational affordances to create an experience for a user. Understanding things like collision shapes, functions that need to be called and received and how illusions of movement are created in games has given me a new appreciation for the work that goes into making games.
One of the challenges I am looking forward to next term is of being able to finish works to a more refined standard. After the course, I would like to be able to continue working towards having a career as a creative practitioner and the work that I do next term will probably set the tone for what this career might look like.
Seeing the work of Kristina Tica, our final guest lecturer, I have been inspired to start thinking about how I might also develop a career that uses practice-based enquiry to explore the benefits and challenges of technology in the lives of the artists. I am also interested in bringing a cross-cultural perspective to these conversations to explore ideas around bias and cultural colonialism that Tica discussed in her talk.
It is hard for me not to be worried about the role of creative practitioners in a future where algorithmic efficiency may be preferred to time-intensive practices like drawing and painting. What I took from Tica’s talk was that perhaps the best way to address these worries is to dive into the debate with paintbrush and code in hand.
My intention for the end of next term is to have learnt enough code to be able to work with cost-effective, environmentally and ethically sound algorithms that also help push my practice into a domain that is more obviously technological. I hope that this will allow me to find the right opportunities for paid opportunities when the course has ended.