4D Tracker

4D Tracker Overview

Introduction

There are approximately 515 million cases of malaria every year, of which roughly three million cases are fatal [5]. To combat the spread of malaria, often the carriers (mosquitoes) are targeted. In the past, mosquito larvae have been attacked with varying degrees of success, but sometimes with notable environmental impact (one solution involves flooding standing water with an film of oil) [6]. Some have suggested a more practical approach may be to target the mosquito’s mating habits. Unfortunately, very little is known about mosquito mating habits [7].

This work proposes to combine contemporary computational photography techniques (4D light fields) and modern tracking techniques (particle filters, model fitting) to measure and eventually, model the mating habits of mosquitoes. In the past, modern tracking techniques have successfully been used to track multi-agent biological systems [2] though only in two dimensions. This work will make use of novel 4D light field sensors to extend previous tracking efforts into three dimensions.

The key contribution of this work is that in previous tracking work using lightfield sensors, the lightfield was rectified and rendered into traditional images. In this work, raw data from the lightfield sensor is fed into the tracking algorithm which reasons about the location of the tracked object in the lightfield domain.

Approach

Previously, to track an object in three dimensions, multiple viewpoints – and thus multiple cameras – were employed [3]. In this work, we propose a single camera three dimensional tracker.

4D Tracker Algorithm

In an image of a typical scene taken by a traditional camera, each pixel is formed by the supposition of all rays emitted from some point in an image. As a result of the supposition, the angle of incidence the ray strikes the image sensor is lost [4]. In a 4D light field camera, the angular information contained in each ray is preserved. These rays are then typically resorted to move the viewpoint of the observer, alter the projection of the scene, or change the depth of field of the image [8]. In this work, the angular information contained in each ray would be used to track objects in three dimensions.

The raw output of the sensor does not form an intelligible image – to a human. The proposed system would form tracking hypotheses based on raw sensor data. The tracker would examine that data and score it based on how well it matched some appearance model. The most likely rays would then be re-sampled based on their previously determined weights. Finally the estimated location of the target would be computed based on the mean of the weighted samples. In the next frame, the rays would be projected using a kinematic model of the mosquito, and are scored and re-sampled again [1].

Preliminary Results

XY plot on left, orthographic plot on right.

The figure indicates the fidelity of the tracker is quite good in the XY plane, though, as expected, the accuracy suffers in the Z axis.

References

1. M. S. Arulampalam, S. Maskell, N. Gordon, T. Clapp, D. Sci, T. Organ, and S. A. Adelaide. A tutorial on particle filter for online nonlinear/non-gaussianbayesian tracking. Signal Processing, IEEE Transactions on [see also Acoustics, Speech, and Signal Processing, IEEE Transactions on], 50(2):174–188, 2002.
2. M. Egerstedt, T. Balch, F. Dellaert, F. Delmotte, and Z. Khan. What are the ants doing?: Vision-based tracking and reconstruction of control programs. 4:4182, 2005.
3. R. Hartley and A. Zisserman. Multiview geometry in computer vision, 2000.
4. M. Levoy and P. Hanrahan. Light field rendering. pages 31–42, 1996.
5. D. J. Rogers and S. E. Randolph. The global spread of malaria in a future, warmer world, 2000.
6. Ronald Rosenberg. Malaria: some considerations regarding parasite productivity. Trends in Parasitology, 2008.
7. Willem Takken. Mosquito mating behavior. 1999.
8. A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin. Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM TRANSACTIONS ON GRAPHICS, 26(3):69, 2007.