NAB Yourself is an interactive web experience designed for self-reflexion and for people to start think about what they really want in life.
Combining facial analysis and speech recognition, people could scan their face and give answers on topics designed to trigger a conversation. While answering, the web application analysed what people said and extracted key themes.
By the end, people had a one of a kind artistic representation of themselves. A unique visualisation of that person’s real ambitions, transforming them into art through the use of technology.
My role
As Technical Director, my role involved to:
- Liaise with both internal and external stakeholders to help with creative, scoping, prototyping and client presentations.
- Design and prototype around feasibility and creative execution. Trying to push boundaries on what was achievable from a technical standpoint, all happening from the browser.
- Lead a team of x3 developers and a tester to build, test and deploy the project under a super tight production timeline.
- Work on the foundations for the core biometrical analysis, speech recognition analysis and real-time artistic illustrations generation.
Key results
- Customers spent an average of 4.25 minutes talking to themselves, with 4 out of 5 users interviewed saying the experience had a positive impact on thinking about what they really wanted in life.
- 41,343 data points generated, allowing the National Australia Bank to connect customers with products tailored to them.
Technology
Biometrical analysis was built upon Amazon Rekognition and a set of custom algorithms, allowing to extract key face metrics, balance the results based on facial expressions and transform properties ratio to align with creative direction.
Speech analysis used both the native Web Speech API, and IBM Watson Cloud services to convert people’s voices into transcripts. We then analysed text in real-time using natural language processing to extract main themes and metrics, transforming what each person said into animated illustrations. Symbols, shapes, positions, and colours of the patterns that fills up the different part of illustrated faces, were driven by voice data.
Working with a national bank means high-standards around security, accessibility and browsers support. To make the entire experience fully AA accessible and support Internet Explorer 11, we’ve had to be creative around solving certain issues. For example, we’ve developed a bespoke feature fallback (based on the Adobe Flash Player) for browsers that didn’t support the MediaDevices interface.