Synaesthesify

Introduction

Shortly after I joined Fjord, I was suggested to give a talk at Fjord's biggest annual event – Equinox – to share a Creative Technologist's point of view towards the design world.

I teamed up with two other new-joiners from Fjord Milan studio – a Visual Designer and another Creative Technologist – to quickly ideate on topics and concepts to talk about. We agreed that data and APIs are rising topics, and it would be inspiring for designers to show what we can do with them.

We came up with the idea of experimenting with Spotify's Track Features API to generate some artworks. Rather than giving a too techy talk, we wanted to engage designers in a more friendly way by producing visual outputs.

Approach

The main idea of this project was to play around with data to generate interesting visual outputs. I also personally wanted to generate something that makes sense, rather than being too artistic. I decided to start by making a chart with available data in order to determine which track features would be useful.

After some trials, I decided to use seven track features shown above to generate a radar chart, with HSL colour extracted from song's key, loudness and tempo. These features are the ones directly related to song itself, which also means they are the most useful to visually find out how a song feels like.

At this point, I could already imagine an artwork generated by scattering the shape of radar charts. Also, by stacking them together in one place, I would be able to easily compare multiple song's characteristics at a glance.

The first outputs I got were close to what I had expected. I tried to fetch track features from all songs in Global Top 50 chart and generate above visualisations. From here, I started to experiment further by tweaking some parameters.

I was inspired by the fact that alignment of each radar charts makes significant difference in the output's look and feel. I quickly added options to toggle randomisations for each chart's position and rotation to see the output.

As can be seen above, these simple tweaks already generated completely different effects, which was something I could not expect. The shapes of radar charts showed some similar patterns throughout the Global Top 50 chart (as a note, this does not necessarily mean these songs are similar; the pattern may come from different scoring system for each feature) that gave the feeling of directional movement in the left example.

Similarly, the charts formed a nice radial pattern as shown on the right example, giving somewhat joyful and playful feeling.

As I had limited time, I decided to stop here and proceed to make my final product – a visual synthesiser for Spotify albums.

Outcome

I took the physical grid sequencers as a metaphor to my app. I already had some design directions suggested by the visual designer, so I could quickly proceed with my idea.

In the final design, users can generate an artwork from a selected album or playlist, and also play around with positions and rotations. Each song in an album is represented as a grid title as shown above, which users can click to check song's features.

This work was presented in a pop-up session at Fjord's Equinox event, together with the Milan team with their own version of experiments. We shared our learnings and initiated conversations around the topic with designers from all over the world.

Learnings

While this mini-project was a simple and quick one, it triggered many discussions in the talk. It became a good opportunity to share experiences from both designers' and Creative Technologists' points of view.

Many designers were interested in collaborating with Creative Technologists but found the process somewhat difficult. Also, as a team of new joiners, we were also keen to find out what has been the process in other studios and what we could suggest them to improve it.

At the end of the talk, one designer from Fjord London studio asked what designers could do to help Creative Technologists; and our suggestion was that they could familiarise themselves in reading developer documentations on APIs to learn what data is available. This will not only help us build our logic but will also become food for thoughts for designing new products and services.