This section will sum up the contributions made in this thesis.
• Survey on thermal cameras and applications. As an introduction of thermal cameras to the computer vision community, chapter2explains the nature of thermal radiation and the technology of thermal cameras.
Moreover, a thorough survey of applications of thermal cameras is pre-sented.
• Detection of people in thermal images. In chapter 4 and 5 algo-rithms for detecting individual people in thermal images are presented.
This includes methods for splitting and sorting detected objects.
• Robust counting of people. The counting method presented in chap-ter 5 includes temporal information on the transition in occupancy to account for noisy detections.
• Sports type classification. Chapter 6 and7 introduces two different methods for classifying sports types performed in the same arena. One method based only on frame-based position of people, while the other method is based on features extracted from short trajectories.
• Tracking applied to thermal video. Few works have been designed for and tested on thermal video. In chapter 8 a real-time multi-target tracking algorithm is presented and chapter 9 introduces a method to improve tracking performance by constraining the number of trajectories produced by an offline tracking algorithm.
• Demonstration of practical applications of computer vision with thermal imaging. The methods developed in this thesis have been ap-plied and tested in several real world cases. PartVpresents five different applications.
References
[1] R. Gade, A. Jørgensen, and T. B. Moeslund, “Occupancy analysis of sports arenas using thermal imaging,” inProceedings of the International Confer-ence on Computer Vision and Applications, vol. 2, feb. 2012, pp. 277–283.
[2] J. Kapur, P. Sahoo, and A. Wong, “A new method for gray-level pic-ture thresholding using the entropy of the histogram,”Computer Vision, Graphics, and Image Processing, vol. 29, no. 3, pp. 273 – 285, 1985.
[3] R. Gade, A. Jørgensen, and T. Moeslund, “Long-term occupancy analysis using graph-based optimisation in thermal imagery,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2013, pp.
3698–3705.
[4] F. Fleuret, J. Berclaz, R. Lengagne, and P. Fua, “Multicamera people tracking with a probabilistic occupancy map,”PAMI, vol. 30, no. 2, pp.
267 –282, feb. 2008.
[5] R. Gade and T. B. Moeslund, “Classification of sports types using thermal imagery,” inComputer Vision in Sports, T. B. Moeslund, G. Thomas, and A. Hilton, Eds. Springer, January 2015.
[6] R. O. Duda, P. E. Hart, and D. G. Stork,Pattern Classification, 2nd ed.
Wiley-Interscience, 2001.
[7] R. Gade and T. Moeslund, “Classification of sports types from tracklets,”
inKDD workshop on Large-Scale Sports Analytics, August 2014.
[8] G. Welch and G. Bishop, “An introduction to the kalman filter,” Chapel Hill, NC, USA, Tech. Rep., 1995.
[9] R. Gade and T. B. Moeslund, “Thermal tracking of sports players,” Sen-sors, vol. 14, no. 8, pp. 13 679–13 691, 2014.
[10] A. Andriyenko and K. Schindler, “Multi-target tracking by continuous en-ergy minimization,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2011, pp. 1265–1272.
[11] K. Bernardin and R. Stiefelhagen, “Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics,” EURASIP Journal on Image and Video Processing, vol. 2008, no. 1, p. 246309, 2008.
[12] A. Milan, R. Gade, A. Dick, T. B. Moeslund, and I. Reid, “Improving global multi-target tracking with local updates,” in ECCV workshop on Visual Surveillance and Re-Identification, September 2014.
[13] H. W. Kuhn, “The hungarian method for the assignment problem,”Naval Research Logistics Quarterly, vol. 2, pp. 83–97, 1955.
[14] R. Gade, T. B. Moeslund, S. Z. Nielsen, H. Skov-Petersen, H. J. Ander-sen, K. Basselbjerg, H. T. Dam, O. B. JenAnder-sen, A. JørgenAnder-sen, H. Lahrmann, T. K. O. Madsen, E. S. Bala, and B. O. Povey, “Thermal imaging systems for real-time applications in smart cities,”International Journal of Com-puter Applications in Technology, accepted for publication.
66 References [15] E. S. Poulsen, H. J. Andersen, O. B. Jensen, R. Gade, T. Thyrrestrup, and T. B. Moeslund, “Controlling urban lighting by human motion patterns - results from a full scale experiment,” inACM International Conference on Multimedia (MM), 2012.
[16] S. Z. Nielsen, R. Gade, T. B. Moeslund, and H. Skov-Petersen, “Taking the temperature of pedestrian movement in public spaces,” Transporta-tion Research Procedia, vol. 2, pp. 660 – 668, 2014, the Conference on Pedestrian and Evacuation Dynamics 2014 (PED 2014).
Occupancy analysis
67
Large facilities, such as sports arenas and cultural centres, are expensive in both building expenses and maintenance. Knowledge about the use and degree of occupancy of these facilities are therefore of great importance in order to optimise the use and include these experiences when planning on new facilities.
Traditionally, occupancy analysis of public facilities has been conducted by manual inspections, either by the administrative staff in the buildings or by external observers. Hiring external observers to watch the activities in each facility over long periods is very expensive. The alternative solution of random checks during a day might be imprecise and misleading for the true occupancy pattern. The work presented in this part suggests two automatic systems based on thermal video feed. A camera can capture data day and night without significant extra expenses. Using the computer vision methods developed in this section the occupancy of a sports facility is automatically estimated.
This part is the first of four main parts of this thesis and includes two pub-lished conference papers:
Rikke Gade, Anders Jørgensen and Thomas B. Moeslund, “Occupancy Analy-sis of Sports Arenas Using Thermal Imaging,”Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), pp. 277–
283, February 2012.
Rikke Gade, Anders Jørgensen and Thomas B. Moeslund, “Long-term Oc-cupancy Analysis using Graph-based Optimisation in Thermal Imagery,” Pro-ceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), pp. 3698–3705, June 2013.
Occupancy Analysis of Sports Arenas using Thermal Imaging
Rikke Gade, Anders Jørgensen and Thomas B. Moeslund
The paper has been published in the
Proceedings of the International Conference on Computer Vision and Applications (VISAPP), pp. 277–283, February 2012.
c 2012 SciTePress
The layout has been revised.
Abstract
This paper presents a system for automatic analysis of the occupancy of sports arenas. By using a thermal camera for image capturing the number of persons and their location on the court are found without violating any privacy issues.
The images are binarised with an automatic threshold method. Reflections due to shiny surfaces are eliminated by analysing symmetric patterns. Occlusions are dealt with through a concavity analysis of the binary regions. The system is tested in five different sports arenas, for more than three full weeks altogether.
These tests showed that after a short initialisation routine the system oper-ates independent of the different environments. The system can very precisely distinguish between zero, some or many persons on the court and give a good indication of which parts of the court that has been used.
4.1 Introduction
In the modern world jobs are becoming ever more sedentary and less physically demanding. This leads to higher demands for activities in people’s spare time, which puts a still growing pressure on the sports arenas. From 1964 to 2007 the number of athletes has quadrupled with a steady increase [1]. Surveys also show that people are dropping the classic club sports in favour of more flexible sports [2]. This calls for a better and more optimal use of the existing sports arenas to keep up with this growing trend.
In order to improve the utilisation of a sports arena, its existing use must be examined. This includes examining the number of users using the arena at the same time and the occupancy of the court. Administrators are especially interested in whether the arena is empty, used by a few people or full and the time for when the occupancy changes. The position of the users is also important as they might only use half a court, which means the other half could be rented out to another group. Manual registration of this would be cumbersome and expensive and an automatic approach is therefore needed.
For such a system to work in general it should be independent of the size of the court, lighting conditions and without any interaction with the users. This can be obtained with a camera.
Detecting people with a camera raises some privacy issues though. Not all people like surveillance and the fear of being observed could keep some people out of the arenas. This work therefore proposes an automatic method to analyse the occupancy of a sports arena using thermal imaging. One of the advantages of thermal cameras is that the persons recorded cannot be identified, which is an important factor if the system is to be accepted by the users of the sports arena. On top of that, thermal cameras are invariant to lighting, changing backgrounds and colours, which make them more desirable for a general application.
74 Chapter 4.