Suche
Close this search box.
Suche
Close this search box.

GongPong.SAT: Developing Realtime Android Apps

The application underlying this case study is at the core of an award-winning concept of a distributed multi-player performance called Music for Gongs and Sateleittes. In short, the vision aims at turning commercially available mobile phones into devices for coordinating a group of performers who are acting as a distributed mass. While the application itself may not be of much practical use to most people, it serves perfectly as archetype of an Android app that makes decisive use of real-time features like system clock and GPS sensor modern portable devices are equipped with. In this paper, we shall explain how to design, organize and implement such an app in Java on the basis of the Android Studio IDE.

The Project

“Music for Gongs and Satellites” is a participatory work for a random public audience that can be performed in any given public space. The basic idea behind the work is to create a situation in which participants can “run wild” within a system while undertaking performative actions, but remain coordinated by some higher power regardless of where or how they “run” within the system. Each participant is given a paired set of tuned gongs and asked to download the needed app to their mobile phones. Participants are readied in about 5 minutes and performing is accomplished through a process of  learning-by-doing. Performances of the work have lasted several hours, but participants can play for a few minutes or a few hours

The Application

The application uses satellites and the mobile phone´s system clock to control an intelligent algorithm that in turn coordinates and instructs the participants via optical marks, based on where they stand, how, when and where they move, as well as how quickly and often they do so. Using a tool based on satellites, the application makes it possible to perform the work in any open space without constraining the dimensions of the area in which the work takes place. In principle, the group of participants could be distributed around the globe and still perform as a whole to create ever changing patterns of beautiful gong sounds that are spread out across an open space, which are paired with choreographic movements, and highlighted by visual impact of the performers.

A simple version of the work is discussed in this paper and in more technical detail. Our example performance takes place in a circular playground area that is divided up into three concentric “zones” around a common center point called “ground zero”. Depending on the zone and the location within the zone that a performer is standing, the controlling app changes the timing of the signalling mark by adding a slight amount of delay or by skipping beats entirely.

Design and Implementation of the Application in Java on Top of Android Studio

The development of the GongPong.SAT app in Java on top of the Android Studio IDE platform provided for some specific challenges. Among these are the need to operate in real-time in a precise rhythm and in total synchronicity between all companion players, the need to know the exact location of a player at any time, and the desire to give useful optical feedback to them.

Receiving and processing GPS data from satellites is at the core of this app. With the goal of providing this in a simple as possible application with a minimum of code, we decided in favor of using Google´s Fused Location Provider API, where “fused” refers to an implicit handling of the source of the location data from GPS satellites or earthly network whichever is available and more appropriate at the place at which the location data is needed.

Moving forward, we will explain of our use of the API in detail: We first observe that any location service code running on top of Android’s operating system must be guarded by a statement ensuring compatibility with the application’s permissions, as they are specified in the application’s manifest file AndroidManifest.xml:

Basically, starting the Fused Location Provider service consists of calling two constituents of the API: a) Getting an initial location fix and b) Launching a task listening for location updates. The following two sections will give the details on both calls..

Get an Initial Location Fix

The following code for getting an initial location fix reveals the asynchronicity of the model: Two listeners are supposed to call (back) a method in case of success and of failure respectively:

Starting a Listening Task

In the case of having received a valid location fix by the onSuccess method, a listening-and-updating task is launched with the following code:

Obviously, the decisive part of this piece of code for task-launching is the installation of the function LocationCallback, which periodically calls (back) the onLocationResult method with a list of newly computed locations, governed by the priority and frequency specifications in the LocationRequest object req. The callback itself first causes a conversion of the list of newly received locations into an array and subsequently sends a call to the UpdateWindow function, whose name refers to a sliding window of distances between the performer´s position and the initial “ground zero” position, computed from location data received via recent callbacks:

In technical terms, the sliding window serves as a basic data structure connecting the two major tasks of this application, a) delivering (“producing”) new location data and b) their processing (“consuming”) that we shall discuss in the following sections.

However, we should not lose sight of the bigger picture and the final purpose of our app: Turning a mobile phone device into a tool for performing a work for gongs and satellites, which is largely a user interface topic. After some initial experiments, we decided in favor of a simple, but efficient, graphical user interface, which is created by switching a circular spot through different sizes (from small to large) and various colors (from yellow to orange and red) at predefined time intervals. The changes in color and size alert the performers when it is time to play i.e. hit the gong.  As the following helper Java class called Signal shows, the user-interface is controlled by another runnable (Notify) whose task is to force a redraw-event for the spot at the next stage by invalidating its current representation:

The full picture of the user-interface is revealed by the app’s activity_main.xml file: a linear layout specifying three elements in vertical order: a text field (for logging), a button (for restarting the app interactively) and the circular signal just explained (for alerting):

It is now time to come back to the consumer task that defines the operating rhythm of the app by periodically processing the data provided by the sliding window and implementing the actual performance logic. This task is implemented by a scheduler (of type Handler) posting a runnable piece of code called Check that is reposting itself to be run again after each second, according to the device´s clock.

Here is the framework code:

The performance logic still to be filled into the body of the run method roughly comprises two functionalities: a) computing the current distance from ground zero and b) deciding about possible actions to be performed depending on the time and the performer’s current position within the playground.

 

Fig. 1. A diagram showing the design for the GongPong.SAT app.

 

Computing the Current Distance from Ground Zero

Early experiments showed that computing distances from individual location fixes unavoidably leads to distorting jumps caused by noisy GPS data. Consequently, we added a simple algorithm for computing a smoothed-out approximation of the current distance from ground zero. At this point our sliding window comes in handy as it supports, for example, a solution based on dropping a few outlier values (in comparison with the previous smoothed-out distance value) and averaging across the remaining recent distance values:

Deciding about the Next Action to be Performed

Instructing players via graphical feedback about the next action to be performed is at the core of the performance logic. As can be seen from the piece of code below, the decision if or not to alert a player (via posting sig.Notify after a slight time shift depending on the player´s relative position within the playground) is based on the player´s zone (circle of radius r0, r1 or r around ground zero) and the multiplicity of ticks (seconds since the start of the performance):

The one final challenge that remains is provided by the request of precisely synchronizing the devices of a (potentially large) number of GongPong.SAT participants. As it is demonstrated by the following code excerpt, we have overcome this challenge quite elegantly by making use of the universal GPS time when launching the checker task of each device at the next full satellite second:

In concluding our presentation we mention again the built-in option of interactively restarting a performance, which allows the app to be recalibrated at ground zero. This is achieved simply by pressing the restart button of our user-interface:

Summary

By taking a media art work as a case study, we demonstrate the feasibility of developing custom-tailored software for off-the-shelf commercial mobile phones, including the use of all their sensors like GPS module and system clock, and for the use as a real-time controller. This is a highly promising result, in particular as it is by no means limited just to creating applications for the arts.

 

Acknowledgment

I would like to thank Art Clay for inspiring me to dive into this adventure at all and my brother Ernst for numerous extremely valuable comments and hints. Further thave tnks to the Swiss Arts Council for providing the needed funding in support of the work and for PHONOS Foundation, Barcelona for hosting the premier of the work during their Fall Festival.

 

 

SI Logo

The Swiss Informatics Society SI is the association of Informatics professionals in Switzerland.

Die Schweizer Informatik Gesellschaft SI ist der Verband der Informatikfachleute in der Schweiz.

La Société Suisse d’Informatique SI est l’association des professionnels de l’informatique en Suisse.

La Società Svizzera di Informatica SI è l’associazione dei professionisti dell’informatica in Svizzera.