A visual surveillance system must be able to detect and track people under a wide variety of environmental and imaging conditions; and it must then analyze their actions and interactions with one another and with objects in their environment to determine when "alerts" should be posted to human security officers. There are many factors that complicate the problem of detecting people from a stationary camera against "fixed" backgrounds, including changes in illumination conditions, background movement due to wind load, or changes in weather. The research reported here developed a novel approach to background modeling and model adaptation that deals effectively with these sources of variation and implemented a real time version of the algorithm that can detect people against complex backgrounds and under changing environmental conditions. For surveillance over large areas, it is unlikely that a surveillance system will have sufficient cameras to monitor the entire surveillance area at high resolution at all times. Instead, the cameras must be multiplexed to obtain such coverage, i.e., scanned over either regular or activity-dependent paths to detect, track, and analyze human activity. The current research developed a control model similar to those used for resource allocation in computer systems to determine when and where cameras should look to maximize the number of targets that can be detected and tracked. Once a person is detected and tracked with a surveillance system, that person's behavior must be analyzed. This often requires analyzing the interactions a person has with other people and with objects. In particular, it is important to determine whether a person is carrying an object and to be able to visually separate the object from the person carrying it so that it can be analyzed by other vision algorithms (e.g., is it a gun or a broom?). The current research has extended and improved on previous research to enable the surveillance analysis to identify a wider variety of objects than under previous conditions. This report provides detailed explanations of these research achievements and provides examples of their utility. Extensive research and illustrations and 4 references