Defense applications white paper

Defense applications white paper

• 1.
  A piXlogic WhitePaper Sponsored by Flex Analytics 4984 El Camino Real Suite 205 Los Altos, CA 94022 T. 650-967-4067 info@piXlogic.com www.piXlogic.com Intelligent Image and Video Search for Defense Applications Government Reseller for piXlogic 10314 Thornbush Lane Bethesda, MD 20814 (301) 787-2989 Jul-2012
• 2.
  July 2012 pg. 2
• 3.
  Contents Introduction Images and videos have always been key Introduction 3 elements of intelligence and defense Problem Statement 3 operations. In recent years, the scope and diversity of digital imagery has greatly Previous Options 4 increased in every area: ground, satellite, The piXserve Solution 5 UAV, surveillance, broadcast, etc. The Key Features of piXserve 5 volume of material being acquired and Security Applications 8 stored is staggering, with no visible plateau in sight. Traditional methods of organizing, Implementation 9 cataloguing, and distributing this material to Summary 9 analysts and the war-fighter are becoming About piXlogic 10 impractical due to the scale involved. On the other hand, timely access to nuggets of vital information contained in images/videos is key to operational success. The ability to cross-correlate the information, whether it’s being obtained from live sources or from archived repositories, is more important than ever. In this environment, image/video search and retrieval has become the new “must have” element of any comprehensive solution. Unfortunately, today’s image/video management systems are not well suited to help make sense of the data collected, and can only provide, at best, very limited search and retrieval capabilities. Problem Statement Most video management systems offer limited options for automating processes such as searching archived footage, or generating alerts from live video. For the most part, these features are either not available, or only available in a very limited sense. Often, a significant amount of manpower is required to carry out even simple search tasks. This is well known in the field. Correlating visual data from July 2012 pg. 3
• 4.
  different sources isanother very generally in the same location on the image. challenging task, mostly done Both of these requirements limit the scope manually today. Automated change of applications possible with such systems. detection is yet another largely elusive goal. Face Recognition: Much as in the ALPR case, a big hurdle is to know where the face Industry/government efforts during to be measured is on the image. To solve the last few years have focused on this, typical systems require that the distance building infrastructure and have between the camera and the subject be resulted in great improvements in the within a predefined range. Lighting ability to acquire higher resolution variations are also critical which is why the imagery/full motion video, moving more successful implementations are limited this material around the network to indoor, entry-way, type of set-ups. efficiently, and storing it. These are Outdoor video in unconstrained great accomplishments, but by environments presents a challenge that is themselves they are not enough. outside the realm of most commercial Now is the time to leverage previous solutions available today. investments and provide a much needed level of automation so that Object Detection: The ability to analysts can deal with the size and detect/recognize/search for specific objects scale of the problems they face. in a video or an image is not usually However, for most solution available. Some attempts have been made providers, this remains a significant for video, but the methods used are overly technical challenge. simplistic and unreliable. A typical technique relies on “frame differencing” to Previous Options separate moving things from a stationary When automated video analysis tools background. The idea is simple but are available, they tend to be single- unfortunately it only works in trivial purpose with limited scope of situations. If the camera is moving, the applicability and stringent operating background will move as well and frame requirements. Consider the differencing techniques won’t work. following three examples: Turning off a light, a cloud passing in the sky, a moving shadow, these are all things Automated License Plate that can yield undesired results. Even when Recognition: For most systems, the the background and the camera are hurdle is to know where the license stationary, the amount of information that is plate is in the image being analyzed. obtained is limited. If the camera is To circumvent this problem, solution calibrated, some guess about the size of the providers either require the use of object can be made and from this an specialized cameras (infrared) or that inference can be derived about what is in the the cameras be placed such that the scene (perhaps an adult, may be not a dog), license plate to be recognized is but even this too can be quite unreliable (is July 2012 pg. 4
• 5.
  it a dog,a tumbleweed, or a far away person). Crowded environments present a critical challenge to today’s 2. piXserve reasons about what it "saw" systems. in the image and develops an initial level of "understanding" about content and context. Where it can, it The piXserve Solution automatically creates searchable piXserve is a general-purpose "tags" for what it saw in the image image/video search and alerting (piXlogic calls these tags "Notions"). solution. Breakthrough technology For example, it can detect the developed by piXlogic allows the presence of things such as: sky, software to automatically “see” the vegetation, flower, face, building, contents of an image/video frame car, map, airplane, helicopter, etc. and create a searchable index and uses this information so that users 3. piXserve uses all the information can search and create alerts in a very calculated from the image to make natural and logical way. comparisons between a search image and previously indexed 1. piXserve automatically images/videos so that users can find “segments” an image in a results that most closely match what way that discerns the they are looking for. individual objects in the image. It creates a 4. piXserve can "see" not only visual mathematical description of objects but also text strings that may the appearance of these appear anywhere in the field of view objects "on the fly", and of the image. This text is also stores it as a searchable index indexed and made searchable. in a database. piXserve works with text from many languages (alphanumeric/latin- character based languages, Japanese, Korean, Chinese, etc.) 5. Depending on the quality of the imagery involved and the type of search being done, piXserve has been designed to achieve accuracies in excess of 85%. Key Features of piXserve  Automatic Indexing Point piXserve to a repository of July 2012 pg. 5
• 6.
  images/video files orto a live the mouse to point to an area of video feed, and automatically the query image to indicate index content. No manual which specific item(s) should intervention or data entry be searched for. required. 3. Browse the contents of existing  Powerful Search databases, grab a frame “on the Through a web browser fly” from a video that is interface, users login to playing, and use that frame to piXserve, connect to formulate a visual search available databases and query. formulate search queries to retrieve desired images/video 4. Search images and videos by segments: object class ("Notion") 1. Use an arbitrary image 5. Type a text string to search to search for pictures/videos where that images/video segments string appears in the field of that contain the same or view (a license plate, a street similar items sign, a name tag, etc.) 6. Search for faces of specific individuals 7. Perform not only simple but also complex multi-modal searches. (Example: find video sequences where something like the bag in this picture AND this face from this other picture AND this text string I just typed all appear in the field of view at the same time.) Use AND, OR, and NOT operators to combine up to 6 criteria in a single query. 2. U s e July 2012 pg. 6
• 7.
  8. Search byfile name and naming) 9. Search by keyword or  Powerful Alerts other external metadata, Create alert criteria just as you would if available. formulate a search query. piXserve- ALERT keeps track of what 10. Submit sample images piXserve machines on the network of non-deformable are indexing and when a match is objects of interest and made consistent with what the user automatically tag specified, it generates a signal. The images/video frames user receives an e-mail with a link to when these items are the alert results. A JMS (Java visible. Messaging Service) signal is also generated to pass the alert on to other  Powerful Automated Tagging systems and applications for further action. 1. Automatically tag images/video frames  Powerful Metadata with the name of The richness of metadata calculated recognized by piXserve about each image/video individuals that frame processed (objects and tags), appear therein can be exploited to enable (automated face customized applications that are of naming). high value in a variety of settings such as: 2. Suggest keywords to describe the contents 1. Automatic determination of of a picture/video change detection when frame (automated videos taken at different keyword times from different angles recommendations) are compared. 2. Determining which portions 3. Submit sample of a video archive contain images of non- useful information, and deformable objects of which could be safely deleted interest and to minimize storage automatically tag requirements. images/video frames when these items are  Scaleable Architecture visible. (automated piXserve is a multi-threaded, J2EE 2D-object detection scalable application that is suitable for the most demanding July 2012 pg. 7
• 8.
  implementations. piXserve extends the capabilities of today’s systems by adding the ability to  Web Services API automatically analyze the video that is being A REST-based API package collected and stored. These video streams is available to support can be intercepted by piXserve and analyzed integrations with third party for alerting purposes. Similarly, recorded applications and workflow video can be analyzed, searched and environments. correlated using piXserve. The analytical capabilities in piXserve support: facial Security Applications recognition, general purpose object If you are concerned with the cost, detection and recognition, text recognition, speed, and accuracy of your video license plate recognition, automatic tagging, investigative work, whether it be and more. All the indexing work is done forensic in nature or dealing with automatically, server side, in the live situations, then you should background. Users are then free to create consider piXserve as a “must-have” visually-based search criteria and navigate add-on to your current system. the body of accumulated material. They can do all of this “on the fly”, as they see fit at Conventional systems focus on the moment, based on whatever problem or managing and manipulating cameras situation they are dealing with. and storage devices. Unfortunately, they only provide limited capabilities The piXserve search environment is for searching the captured video: intuitive and productive, and the user time, date, motion, transaction interface is through a web browser (Internet trigger…these are among the more Explorer, Mozilla Firefox, Safari, Google common set of options available. Chrome, or equivalent). Users can drag- While useful, these features alone are and-drop a picture from anywhere to inadequate to support a productive formulate a similarity search query, or pause workflow and significant manpower a video while it’s playing, and use that effort is required even for the simpler frame to create a new search criteria or tasks. Common situations involve refine an existing one. This latter capability several operators having to stare at a greatly simplifies the discovery process bank of monitors for hours on end in precisely in those situations when the user order to catch an event of interest, or isn’t quite sure what they are looking for and having to wade through hundreds of are working in an investigative/exploratory hours of video from many cameras mode. looking for a specific event or trying to correlate separate ones. These situations are labor intensive, error prone, and do not scale well. July 2012 pg. 8
• 9.
  Implementation piXserve can processvideos in a variety of formats (MPEG-1, MPEG- 2, MPEG-4, H-263, H-264, etc.). piXserve can also process still images in over 90 different formats (jpeg, tiff, png, bmp, psd, etc.) piXserve can index both archived video as well as live video broadcast from Multicast IP cameras. piXserve and piXserve-ALERT run on standard 2-CPU rack servers (multi- core Intel-Xeon processors or equivalent), in a Windows Server 2003 or 2008 environment. Customers typically choose Dell or HP hardware for implementation. parallelize throughput and serve growing piXserve is available in both x32 and needs. x64 bit versions. In order to index archived video The metadata created by piXserve is stored piXserve requires that the storage in an RDBMs (Oracle or MS-SQL are device be accessible via a network supported, PostgreSQL is bundled with share (Linux/Unix/Windows). piXserve). The data and the piXserve output Further, the stored video should not can be integrated/correlated to that from be in a proprietary, non-standard other systems that the customer may be format. using. The alerting functionality is provided by piXserve-ALERT. A single instance of A single server can process large piXserve-ALERT can serve many users and amounts of archived material, or live monitor potentially thousands of alert video from multiple feeds/sources. criteria. Here too scaling is achieved by The higher the number of cores on adding additional ALERT servers. In the server, the higher the number of configurations were several hundreds or hours of video per day that can be thousands of individuals will be searching processed by a single machine. piXserve generated data, the use of piXserve implementations can range piXserve-Enterprise Edition is in size, from as little as a single recommended. server to scalable multi-server and distributed configurations. The Summary architecture of the product is such Images and videos are a critical element of that as the needs of the customer defense and intelligence operations. It is grow, hardware can be added to very difficult to deal with an ever-growing amount of captured video without July 2012 pg. 9
• 10.
  automation. The alternativesto  Content Auto-tagging (automatically automation are expensive, time label an image/video) consuming, and prone to errors. At  Content Alerting (automatically inform the same time, there is a lack of users when items of interest appear in a suitable tools to provide a live video stream or web crawl) meaningful level of real-world automation.  Content Change Detection (automatically compare images and piXserve provides an unparalleled video segments to detect changes at level of image analysis and the object level) understanding. In a single tool it provides capabilities that span: piXlogic serves the needs of government object detection and recognition, and industrial customers. piXlogic sells its face recognition, license plate products directly and through a network of recognition, text recognition, resellers in the US, the UK, Japan, Australia, automatic tagging, and more. In Argentina, Israel, and Italy. each of these areas, piXserve redefines the state of the art and can Corporate help your meet the efficiency and piXlogic, Inc. T. +1-650-967-4067 4984 El Camino Real E. info@piXlogic.com effectiveness goals that you have set Suite 205 W. www.piXlogic.com for yourself. Los Altos, CA 94022 Flex Analytics is a systems integrator and About piXlogic software reseller in the U.S. Intelligence piXlogic is a privately held company Community. It supports the sale, located in Los Altos, CA, USA, the implementation and customization of heart of Silicon Valley. piXlogic is piXserve in government installations. an In-Q-Tel portfolio company (a venture capital organization that Government Sales +1-301-787-2989 Flex Analytics LLC gpepus@flexanalytics.com serves the needs of the US 10314 Thornbush Ln www.flexanalytics.com Intelligence Community). The Bethesda, MD 20814 company’s flagship products are piXserve and piXserve-ALERT. The software enables:  Content Discovery (find pictures/videos that contain specific objects, scenes, text, or people of interest) July 2012 pg. 10