An Interactive Virtual Reality Experiment for storytelling based on Twitter Real-time Data

City of Sparkles is an interactive virtual reality experience using real Twitter messages to tell stories of New York City where an artistic virtual New York City is dynamically shaped with sparkles. Each of sparkles represents a real Twitter message from residents of New York City, and its color and motion represents the emotion mood and the geographical location of the message. Audience can freely fly in the virtual city, and discover the messages left in the city, and feel the lonely souls.

The experience contains several chapters. Each chapter has a story line and its own emotion mood. We use Natural Language Processing to programmatically curate the tweets to match the mood of the chapter. For example, the black and white scene represents anger and anxiety, while the blue scene consists of tweets reflecting lonely but positive emotion. Audience can interact with the sparkles and feel the emotion of the messages. In accordance to the emotion, abstract visual effects and motion are happening as well as adaptive contemporary music and sound, so as to strengthen the feeling of the audience.

Media: Interactive Virtual Reality
Experience Length: 5min ~ 10min
3D Data: collected by aerial photogrammetry
Message Data: collected via Twitter API from Jan 2017 to Jun 2018