TitleImage Surveillance Assistant
Publication TypeConference Proceedings
Year of Conference2016
AuthorsMaynord, M, Bhattacharya, S, Aha, DW
Conference Name2016 IEEE Winter Conference on Applications of Computer Vision (WACV) Workshop on Computer Vision Applications in Surveillance and Transportation
Date PublishedMarch 2016
Conference LocationLake Placid, New York

Security watchstanders who monitor multiple videos over long periods of time can be susceptible to information overload and fatigue. To address this, we present a configurable perception pipeline architecture, called the Image Surveillance Assistant (ISA), for assisting watchstanders with video surveillance tasks. We also present ISA1, an initial implementation that can be configured with a set of context specifications which watchstanders can select or provide to indicate what imagery should generate notifications. ISA1’s inputs include (1) an image and (2) context specifications, which contain English sentences and a decision boundary defined over object detection vectors. ISA1 assesses the match of the image with the contexts by comparing (1) detected versus specified objects and (2) automatically-generated versus specified captions. Finally, we present a study to assess the utility of using captions in ISA1, and found that they substantially improve the performance of image context detection.

NRL Publication Release Number: