OptiMouse: a comprehensive open source program for reliable detection and analysis of mouse body and nose positions
© Ben-Shaul et al. 2017
Published: 15 May 2017
Accurate determination of mouse positions from video data is crucial for various types of behavioral analyses. While detection of body positions is straightforward, the correct identification of nose positions, usually more informative, is far more challenging. The difficulty is largely due to variability in mouse postures across frames.
Here, we present OptiMouse, an extensively documented open-source MATLAB program providing comprehensive semiautomatic analysis of mouse position data. The emphasis in OptiMouse is placed on minimizing errors in position detection. This is achieved by allowing application of multiple detection algorithms to each video, including custom user-defined algorithms, by selection of the optimal algorithm for each frame, and by correction when needed using interpolation or manual specification of positions.
At a basic level, OptiMouse is a simple and comprehensive solution for analysis of position data. At an advanced level, it provides an open-source and expandable environment for a detailed analysis of mouse position data.
Many studies of rodent behavior are based on the analysis of positional data. Such data can reveal place preference, attraction to particular stimuli, or other positional information such as speed and distance covered [1–6]. Analysis of videos of mouse behavior in arenas can be performed by manual scoring or by automated video analysis. While the first approach does not require any dedicated software, it is highly subjective, prone to error, and time consuming if conducted rigorously. Automated analysis of position data is objective and usually fast, but is often also not error free.
The challenge of position detection is not to identify body and nose positions in any single image, but rather to do so correctly for the vast majority of frames in any given movie. This is considerably harder for detection of nose as compared to body center coordinates. This distinction is important because it is the snout that indicates what a mouse is attending to. The difficulty of the image analysis problem varies from video to video. Decreased contrast between the mouse and the background, increased variability in mouse posture, frames that do not include the mouse in its entirety, as well as temporal and spatial inhomogeneity of the arena, are all factors that increase the likelihood of errors. Common procedures for detecting nose positions involve an initial step of automatic detection, followed by a reviewing of the detection algorithm’s performance. When errors are detected, they can often be manually corrected. The last stage can be very laborious and time consuming, making it difficult to detect all instances of erroneous detections. For example, a 10-minute movie with a 30 Hz frame rate, will include 18,000 frames. This is a large number of frames to monitor, let alone to correct.
Here, we describe a new open-source software, OptiMouse, designed for the analysis of positions of individual mice in a behavioral arena. OptiMouse is written in MATLAB and covers the entire sequence of mouse position data analysis, starting from a video and ending with processed measures of mouse position and movement data within an arena. Other programs for automated analysis of position data exist. Some of these are commercial products with a broad range of features (e.g., Ethovision, ANT-maze), others are freely available programs. Among these, several are designed for detection of particular features such as stretch or grooming [7, 8], exploratory behaviors [9–11], or social interactions [12, 13]. Other programs also include advanced algorithms for detecting patterns in behavior [14–16]. The motivation for creating yet another software program for this general problem is two-fold. First, OptiMouse was designed to provide highly accurate detection of positions, and to facilitate the process of reviewing and correcting, when needed, the automatically detected positions. As described below, many features in OptiMouse were designed with the objective of improving accuracy in position detection. Second, OptiMouse was developed with the goal of making the entire analysis process accessible and free. Working with OptiMouse involves graphical user interfaces, and does not require any programming knowledge. A detailed user manual accompanies the code and all detection stages can be controlled by the user.
Although it provides several algorithms for detection, OptiMouse does not rely on any single algorithm to detect positions under all scenarios. The key assumption behind OptiMouse is that, while usually there will not be a single set of parameters (a setting) that correctly identifies positions in all movie frames, a small number of settings will suffice to provide correct detection in the vast majority.
Application of multiple detection settings to a single session. In addition to several built-in algorithms, each with its own associated set of parameters, OptiMouse allows seamless integration of custom functions for body/nose detection into the user interface.
Efficient identification of frames with incorrect position detection. In addition to serially reviewing the video to identify problematic frames, OptiMouse allows simultaneous viewing of various features describing each of the frames. This ability facilitates identification of frames with incorrect position detection.
Methods for correcting detection in frames with errors. This includes the application of settings to single frames or to groups of frames, defining manual positions, and interpolating body and nose coordinates.
The ease of use, expandability, and performance of OptiMouse should make it useful both for occasional investigators of mouse behavior and for researchers that wish to apply a rigorous analysis of position data.
Key stages in OptiMouse
Define arena boundaries
The user must define the region of interest for analysis and provide actual arena dimensions.
Conversion of raw video data to MATLAB files
Once arenas are defined, the user can initiate conversion Conversion can be run in batch mode, allowing definition of multiple arenas and then running conversion for all, in a single operation.
Determine optimal detection settings
The user may simply accept the default algorithm and initiate detection; however, in most cases, detection of body and nose positions can be significantly improved by small adjustments of the detection algorithm parameters and, often, by definition of multiple algorithms. In this stage, the movie is browsed to find detection settings that minimize errors in the detection of body and nose positions. The goal is to define a minimum number of algorithms so that at least one is appropriate for each frame of the movie. If multiple settings are defined, one of them must be specified as the default.
During detection, OptiMouse finds mouse positions in each frame according to the settings defined by the user. Like conversion, detection can be time consuming and thus the option for a batch mode is provided. Detection settings can be defined for multiple sessions, and then run in a single batch operation. Note that the output of the detection stage, the position files, can be immediately used for analysis manually or by using the OptiMouse analysis interface.
Testing, and if needed, correcting, of the detection settings; annotation
If the stage is bypassed, then the default setting will be applied to all frames. The reviewing stage allows overriding of the default algorithm positions. Individual frames or groups of frames can be assigned with any of the non-default settings defined by the user, or set manually. OptiMouse provides multiple tools to browse the video or to locate individual frames with specific attributes. The impact of reviewing on the final position data depends on the number of errors associated with the default detection setting. A major effort has been made in OptiMouse to provide tools to efficiently locate problematic frames. Nevertheless, this stage may be time consuming. During reviewing, it is also possible to add annotation to the video.
Derive meaningful behavioral parameters from the position data
Several analysis options are provided at this stage via the graphical user interface. Additionally, task-specific analyses can be implemented by analyzing the Results file. In this stage, the user can define zones (within the arena) and analyze position data with respect to these zones.
Frequently asked questions concerning the key properties of OptiMouse
What programs must be installed on my computer to run OptiMouse?
MATLAB with the image processing tool box. OptiMouse has been tested on MATLAB releases 2015b and 2016a and 2016b. OptiMouse is not compatible with some older versions of MATLAB which use different coding conventions for graphical interfaces.
Which operating systems can OptiMouse run on?
OptiMouse was developed in Windows 8. However, it should be compatible with other operating systems that run MATLAB.
Do I need to know MATLAB to run OptiMouse?
Running OptiMouse requires a very basic familiarity with MATLAB. At the minimum, it is required to set the path and call the program from the command line. More advanced data analyses, and modifications and extension of the code, naturally require MATLAB programming skills.
Which video files formats can OptiMouse use?
OptiMouse was tested with the following formats: mp4, mpg, and wmv. However, any format supported by the MATLAB VideoReader object should be valid. The list of supported formats is obtained by typing “VideoReader.getFileFormats” on the MATLAB command line. On MATLAB 2016b running on a windows OS, this yields:
.asx - ASX File
.avi - AVI File
.m4v - MPEG-4 Video
.mj2 - Motion JPEG2000
.mov - QuickTime movie
.mp4 - MPEG-4
.mpg - MPEG-1
.wmv - Windows Media Video
Is OptiMouse suitable for real time processing?
No. OptiMouse is an offline analysis software.
Is OptiMouse suitable for analyzing social interactions?
No. OptiMouse is designed for analyzing the behavior of a single mouse.
Which types of behavioral tests is OptiMouse suitable for?
OptiMouse is suitable for any test that involves a single mouse in a stationary arena. Examples of standard tests that fall into this category are place preference tests, open field behavior, plus mazes, three-chamber tests.
Does OptiMouse identify specific body regions?
Yes. A key aspect in OptiMouse is the detection of body center and nose positions. Some of the detection algorithms in OptiMouse also detect the tail-end and tail-base as intermediate stages of nose detection. However, the coordinates of these positions are not used in other analyses.
Can OptiMouse detect particular postures, such as rearing and grooming, or gait patterns?
No. OptiMouse does not provide automatic detection of body postures. Manual annotation on a frame by frame basis is possible in OptiMouse.
What detection algorithms are built into OptiMouse?
OptiMouse includes several detection algorithms, each of which includes several parameters that can be modified by the user. All of them rely on a color contrast (after the movie has been transformed to grayscale) between the mouse and the arena. Most built-in algorithms employ “peeling” of the mouse image perimeter, which allows detection of the tail and then assists detection of the nose. A detailed explanation is provided in the user manual. In addition, OptiMouse allows incorporation of custom written detection functions.
Are there any constraints on the coat color of the mouse?
Yes. The coat color must be distinct from the arena’s background. Ideally, the entire mouse should be darker or lighter than the arena. Thus, a black mouse on a white arena or vice-versa are the ideal scenarios. However, if the mouse contains small patches of a different color on its body, this should not significantly impair detection. Current detection algorithms in OptiMouse will not perform well with a two-colored black and white animal on a gray background.
I have an algorithm that works really well on my videos. Can I use it in OptiMouse?
Yes. OptiMouse is designed to incorporate other detection algorithms. Custom algorithms are “declared” in one of the OptiMouse folders, and then are essentially incorporated into the user interface. Custom functions accept image data and other optional parameters, including user defined parameters. It is even possible to set user-defined parameters (which are not part of the current detection algorithm) graphically via the OptiMouse GUI. Custom written algorithms must, at minimum, return body and nose positions. See the user’s manual for a more detailed explanation of how to write and incorporate custom algorithms into OptiMouse.
How long will it take me to obtain results from a video file?
The answer obviously depends on many factors. Under ideal conditions, which require minimal user intervention, the entire procedure to process a 10 minute video may require about 20–30 minutes. The actual values also depend on video frame rate, resolution, and computer processing speeds. Most of the time is spent on automated processing not requiring any user input. Such processing can be performed in batch mode, so that the actual user time (with minimal user intervention) is a few minutes.
Which stages require most user intervention?
The two most time consuming stages are setting detection parameters and reviewing the video after detection has been performed. In both cases, the required time depends on the video quality. Videos in which the mouse is always easily separated from the arena facilitate both stages. Videos with variable conditions and poor image signal-to-noise ratios will require more tweaking of the detection settings. Setting detection parameters involves browsing the video and deciding on a number of detection algorithms. This typically requires a few minutes for each video (settings can be saved and applied to other movies if they share similar attributes, thus reducing the time required by the user). Reviewing of the movie can be a lengthy process and depends on the performance of the detection algorithms and the desired accuracy. The reviewing stage includes multiple tools to easily identify frames with erroneous detection, as well as frames in which the mouse is in particular parts of the arena, allowing the user to focus reviewing efforts on the frames that matter most.
Can behavioral data be synchronized with other data (e.g., electrophysiological data)?
OptiMouse does not include a built-in synchronizing signal. However, the Results file contains a frame by frame account of various parameters such as body and nose position, body angle, speed, presence in a certain zone, and occurrence of annotated events. The Results file also contains a time stamp for every frame so that, if the first video frame is synchronized with other non-video data, all other OptiMouse values can be aligned as well.
Can I modify OptiMouse beyond the addition of custom detection algorithms?
OptiMouse is stored in Github, with the hope that this will facilitate a community based development of the code. OptiMouse is written in MATLAB and individual code files (m-files) are annotated. The user manual provides a description of file formats, algorithms, and data conversions. The graphical user interface has been designed with the MATLAB GUIDE tool, which can be used to modify the existing interfaces and the code associated with various controls.
Which analyses are implemented in the Analysis interface?
In the analysis stage there is a distinction between zone dependent and zone independent analysis. Without defining zones, OptiMouse can generate graphical displays of positions (as tracks or heatmaps), speeds, and body angles as a function of time. If events have been annotated during the annotation stage, their total occurrence during the session or their distribution in time can also be plotted. In addition, the Analysis GUI allows the definition of zones (of arbitrary number and shapes within the arena). Once zones are derived, the analysis of positions and events can also be shown as a function of zone entries.
Can I implement my own data analyses on positions detected by OptiMouse?
Yes. The Position file contains frame by frame mouse positional data as well as user annotated events. The Results file also contains zone-related information. Both files are MATLAB data files (*.mat) which can be used for more complicated analysis. A detailed description of the Results file format is provided in the user manual. In addition, the user manual provides example code for the analysis of freezing episodes from the Results file.
Does OptiMouse support batch processing of position data files?
No. However, OptiMouse contains an option for adding tags to individual files. These tags then allow grouping of files for statistical comparison of groups of files. See the manual for more details on the use of “experiment tags”.
How do I get more information about OptiMouse?
OptiMouse includes a detailed user manual. For issues that are not covered in the manual, contact the corresponding author of this manuscript by email.
The manuscript describes the conceptual ideas behind the operation of OptiMouse, rather than the practical steps of working with the software. The latter, along with more complete descriptions of the features and algorithms implemented, are covered in a detailed user manual (Additional file 1). Although all stages are described, the emphasis here is placed on the stages that are unique to OptiMouse, namely the detection and the reviewing stages.
Defining spatial and temporal regions of interest
Detection of body and nose positions
Once detected, the mouse body center is simply defined as its centroid. Detection of the nose is generally more challenging, and is addressed by several features in OptiMouse. The first of these is the availability of several algorithms for detection. Most algorithms rely on detection of the tail, which is identified by a process denoted here as peeling, in which the outer layer of mouse pixels is sequentially removed (Fig. 6a5). Since it is the thinnest part of the mouse, the tail is the first part to disappear in this process.
Using the default algorithm, the nose is defined as the furthest point from the tail base, with distance measured along the outer layer of the mouse. Detection using these default settings is shown in Fig. 6a6, with detected landmarks drawn over the mouse image. Of the four landmarks indicated in Fig. 6a6, only the nose and body center are required for subsequent analysis stages (tail base and tail tip represent intermediate stages in the calculation of nose and body positions and are not used further).
Figure 6b shows the results of detection of various frames from the same session, again using the default settings. In these examples, positions are correctly identified for a variety of postures. Generally, however, settings must be adjusted to achieve good identification. The purpose of the Detect GUI is to identify those settings. Two key parameters are the threshold and the number of peeling steps. The effects of changing these parameters are illustrated in Fig. 6c and d, respectively. For both these parameters, a narrow range of values yields good detection. Other parameters that are determined through the Detect GUI include the algorithm used for calculation and the specific background image.
The GUI allows browsing of video frames throughout the entire session, with the aim of finding parameters that provide good detection across most, if not all, frames. Once a successful combination of parameters is found, it can be defined as a setting (in Fig. 5, under the defined settings panel, the setting was named setting 1). The run button in the GUI will apply the setting to find nose and body positions in all session frames. As with the preparation stage, detection is a time consuming process which can be run in batch mode.
Detection of body and nose positions using multiple settings
In addition, OptiMouse facilitates the integration of custom functions into the GUI. Any MATLAB function that accepts as input a frame image and returns as output nose and body coordinates may be integrated into the GUI. User defined functions can accept all built-in parameters as well as additional unique GUI inputs. Details on the requirements and application of user defined functions are provided in the manual (Additional file 1).
In summary, the goal of the detection stage is to identify a minimal set of settings, such that, in each frame, at least one yields correct detection. When detection is run with multiple settings, each will be applied to each of the frames (Fig. 4). The batch option applies to multiple settings as it does for single settings.
Reviewing position detection
The left side of the Review GUI includes buttons that apply each of the settings to the currently shown frame. The images in Fig. 10a show the same frame with each of the available settings applied. In this example, the default (blue) setting does not provide correct detection, but the second (green) and fourth (cyan) settings do. When none of the predefined settings work, it is possible to define manual positions or to exclude individual frames from the analysis.
The second group of actions (Fig. 8) allows application of settings to a continuous sequence of frames. Manual positions cannot be applied to an entire segment in a single operation, but it is possible to assign positions to an entire segment, using interpolation based on the start and end positions (Fig. 10b, c).
The parameter view
The ability to apply settings to an entire sequence of frames accelerates the correction process, but it nevertheless requires serial viewing of frames. A key feature of OptiMouse, designed to further expedite reviewing, is the option to view information about all frames simultaneously. This is accomplished with the parameter display (right side of the Review GUI, Fig. 9). In this display, each frame is indicated by one dot, with the horizontal and vertical coordinates representing features associated with the frame. The currently shown frame is highlighted by the black diamond. Pressing the left mouse button while the cursor is near a dot will show the associated frame, providing direct access to frames with particular attributes. As described in the manual, there are numerous parameter combinations (views) which can be shown. Here, we provide a brief description of some of these views and the operations that can be achieved with them.
The parameter views can show many parameters associated with detection in each frame. Examples include mouse length and area, angle, body and nose positions, intensity values of mouse or background, and more. The interface allows plotting of any of these parameters, as determined by any of the settings, against any other parameter as determined by any of the settings. For example, in Fig. 11b, dots represent the length and mean intensity of the mouse. This representation happens to be useful for identifying a frame with extreme feature values, where the detected object is clearly not a mouse.
In Fig. 11c, the dots represent mouse angles as detected by two different methods. Note that axes label texts describe the parameters shown, while their colors indicate the relevant setting. In this example, most dots are located on the diagonal, indicating agreement between the first and second settings. Off-diagonal dots are more likely to represent erroneous detections, since at least one of them must be incorrect. The purpose of views like those in Fig. 11c is to identify frames with discrepancies between settings. Thus, in the particular frame shown (Fig. 11c, right), the first (blue) and the second (green) settings yield angles that are almost opposite. Here, correct detection is provided by settings 1 and 4, while settings 2 and 3 confound tail and nose.
Figure 11d shows another special view where the dots represent the angular velocity of the mouse. Dot positions indicate the angle differences between the preceding and the current frame. Exceedingly high values are associated with erroneous detection in either the frame itself, or in the one before it. This is indeed the case with the frame shown in the right side of Fig. 11d, where the default setting confounds tail with nose. This and two other related views (body speed and nose speed), are useful for detecting, and as described below, for correcting erroneous detections.
Another useful view (Fig. 11e) shows the active setting associated with each frame. In addition to showing the distribution of the different settings, it allows identification of frames that are not associated with any valid position. These are denoted as NaN (not a number) frames. NaN frames lack positions because their active setting does not yield a valid position. This is the case for the default setting in the frame shown in Fig. 11e. In this frame, three settings do yield a valid position, but only two are correct (settings 2 and 4, as can be seen by examining the frame image).
Operations that can be performed with the parameter view
The importance of the parameter view is its ability to identify frames with particular attributes, as well as to apply changes to a group of frames simultaneously. Some of these operations are described below.
Once frames are marked, it is possible to selectively skip from one marked frame to another, or to pause continuous playback on marked frames. Frame marking can be applied in any of the views, providing numerous possibilities to hone in on frames associated with particular features. Examples include frames with particular parameter values (Fig. 11b), discrepant values between two settings for a given parameter (Fig. 11c), frames assigned with particular settings (Fig. 11e), or frames associated with abrupt changes in position (Fig. 11d). The latter are of particular importance to marking, and include body speed, nose speed (defined relative to the body), and the previously described angle change view. In each of these views, frames can be marked not only using polygons (as in Fig. 12c) but also by setting a threshold value, above which all frames will be marked. If the threshold is sufficiently high, then marked frames, or those immediately preceding them, are likely to reveal incorrect detections.
Applying settings to a group of frames
In addition to marking and applying predefined settings, polygon definitions in the parameter view can be used to exclude frames. Frame exclusion is useful for when frames should be ignored due to technical problems with them.
Correcting brief detection failures
An additional, optional, function of the reviewing stage is to assign events to frames. Annotated events can include any user defined feature such as grooming, freezing, rearing, or any other postural or dynamic property. As indicated in Fig. 8, event annotation can be applied to single frames or to continuous segments of frames. See the manual for a more detailed explanation of annotation.
Analyses can be divided into several categories. The positions and speed panel includes analyses that can be performed independently of any zone definitions (Fig. 16), including tracks made in the arena, position heatmaps, distance travelled, velocity profiles, and angle histograms. Tracks and positional heatmaps are shown separately for nose and body positions.
The events panel provides analyses of the distribution of events in time and space. When both events and zones are defined, the occurrence of events within particular zones can be displayed as well (using the in zones button in the events panel). The complete list of analyses and displays is described in detail in the manual (Additional file 1).
In this section, we briefly describe a few examples that provide some general insights on analysis of position data using OptiMouse. This analysis is based on a two-odor preference test in a rectangular arena, but these principles apply to a broad range of arena configurations.
Figure 17a shows a movie frame (as seen in the Analysis GUI) following the definition of eight zones. In this session, the mouse was an adult female ICR mouse, which sampled two plates with male urine stimuli. The arena was a plastic cage with dimensions 36 × 20 × 18 cm (L × W × H). Lighting was supplied by four red lamps (General Electric, 25 W, LED light bulbs) placed above the cages and positioned to minimize shadows. The video was made with a HD camcorder (Canon Legria HFR106), mounted 210 cm above the cage floor. The video format was a 24 bit per pixel, RGB, with a 1280 × 720 pixel resolution, taken at 30 frames per second. Movie duration was 602 seconds (18,071 frames). Arena preparation (with a single arena defined, see Figs. 2 and 3) required 745 seconds (on a Lenovo T440s laptop with 12GB of RAM, an i7-4600 processer, and a Samsung SSD 850 EVO disk). Detection (Figs. 4 and 5) used four different settings (setting 1 (default): algorithm 6, three peeling steps, threshold 0.5, setting 2: same as setting 1 with threshold 0.8, setting 3: same as setting 1 with threshold 1.35, setting 4: algorithm 7, three peeling steps, threshold 1.75). Detection required 1407 seconds. Detection using a single setting (the default) required 395 seconds. During reviewing of the 18,071 frames, 489 (2.7%), 262 (1.5%), and 1434 (8%) were assigned to settings 2, 3, and 4, respectively. Eighteen (0.1%) frames were assigned manual positions and 383 (2.1%) were interpolated.
Figure 17b and c are displays of body and nose tracks, respectively, for the entire session. Comparison of the two panels underscores the importance of accurate identification of nose rather than body coordinates. Specifically, it is the nose rather than the body that enters the small circular zones containing odor plates, and provides an indication of preference. Figure 17d shows a heatmap display of nose positions. Figures 17e and f show times of zone visits and the enrichment score as a function of time (only nose positions are shown in these panels). The total time spent in each zone, which is proportional to the number of dots in Fig. 17e, is shown in Fig. 17g. The enrichment score at the end of the session, equivalent to height of the traces in Fig. 17f at the last time point, is shown in Fig. 17h.
Examination of the enrichment score as a function of time for the different zones provides two insights. First, the actual preference values are highly sensitive to the precise definition of the zones. In the current example, the small circular zone at the bottom (blue) is associated with stronger preference than the larger circle at the bottom (cyan). Consequently, the difference in preference between the small lower and upper circles is considerably larger than that for the corresponding large circles. Second, the specific time point at which the enrichment score is considered can have a significant impact on the preference scores. As seen in Fig. 17g, in most zones, the enrichment score changes as a function of time. This in itself is not unexpected, but it is noteworthy that zone preference can change abruptly, and in some cases, may even reverse in time. Thus, care must be taken when defining zones and setting the exact periods for analysis, as these may have a profound, qualitative effect on apparent preferences.
OptiMouse does not include a module for population level analysis. However, when the results are saved to a MATLAB file, it is possible to add tags to each session. These tags are useful for identifying sessions for subsequent population level analyses of position data.
In this manuscript, we presented OptiMouse, an open source MATLAB-based code for the semiautomatic analysis of mouse position data. Although most examples above focused on a simple rectangular arena and two-stimulus preference tests, the principles described here apply to other arena configurations and behavioral tests. Because OptiMouse allows definition of arenas and zones or any arbitrary shape, it is suitable for any test that involves a single mouse within a static arena, including tests such as open field behavior, place preference or avoidance, and plus mazes . OptiMouse is not suitable for analyzing social interactions because the detection algorithm is designed for a single mouse in an arena. At the analysis stage, any behavioral parameters that are defined by the body and/or nose positions can be easily extracted from the Results file. This includes measures such as speed, acceleration, and body angle. Freezing episodes can be easily derived from speed velocity profiles. Appendix 9 in the user manual (Additional file 1) includes code for analysis of freezing using the Results file and examples of the analysis. Furthermore, if zones are defined, the dependence of each of those measures on the zone can be easily derived using the Results file which contains occupancy information in each zone for each of the frames. Information about more subtle postural features (such as grooming and reading) can be incorporated into the Results file using manual annotation which allows frame by frame annotation of any user defined event (e.g., freezing, grooming; see the manual). Once defined, the analysis of the occurrence of these events in time and space, and particularly in specific zones, is an integral part of the analysis GUI. Automatic detection of such subtle postural features is beyond the current scope of OptiMouse. Implementing such automatic detection can be achieved via user defined detection functions, which will need to return other variables in addition to the required variables that are returned by default (e.g., body and nose positions). See Additional file 1: Appendix 5 in the manual for a detailed explanation of the outputs of the detection functions. OptiMouse does not support an explicit synchronization signal, but alignment of behavioral data with non-video data (such as electrophysiological data) is possible if the non-video data contains a time stamp that is aligned to the video. Since the Results file contains a variable with the time of each frame and a frame by frame description of all position and movement parameters, events and behavioral features can be compared at the temporal resolution of the video frame rate.
At a basic level, OptiMouse provides a simple yet comprehensive solution for the analysis of position data, starting with video data and culminating with graphical displays. At a more advanced level, OptiMouse is an expandable software with multiple tools designed to achieve high accuracy detection and more advanced analyses of positional data. Using OptiMouse at a basic level is appropriate for videos with good quality, when high accuracy determination of nose positions is not critical, or when the relevant measures concern body rather than nose positions. Under these conditions, after defining arenas, it may suffice to apply one detection setting and directly analyze the data, entirely skipping the reviewing stage. The strength of OptiMouse, however, lies in the ability to achieve high quality nose position determination under less than ideal conditions. This ability involves two elements. First, multiple settings can be defined for any given session (Fig. 4). Second, positions are assigned to each frame based on the settings that yield the most accurate detection (Fig. 8). When none of the predefined settings work, positions can be assigned manually or interpolated.
While setting detection parameters, users have almost full control over all aspects of detection, including background image subtraction, threshold level determination, number of peeling cycles, and selection of the algorithm to apply. Despite this flexibility, and despite the ability to apply multiple settings, the built-in algorithms cannot cover all image detection scenarios. Acknowledging such limitations, OptiMouse is designed to seamlessly integrate user defined functions into the interface as if they were built-in.
Indeed, the simplest approach to expansion of OptiMouse is to apply custom detection algorithms. Custom algorithms may be slight variations of built-in algorithms, but can also implement entirely novel approaches. They may also use other parameters in addition to those integrated within OptiMouse. It should be noted that the algorithms that can be most easily integrated are limited to single frame analysis of grayscale converted images. Incorporation of color information, usage of prior knowledge in any given frame (e.g., using Kalman Filters), and automatic derivation of other positional features (e.g., automatic detection of rearing, grooming [7, 8]) are potential improvements that will require more extensive modification of the code.
When it is important to minimize detection errors, the reviewing stage is crucial. One way to review a session is to watch it in its entirety, pause on errors, and correct those using predefined settings or a manually defined position. If there are many detection failures, then serially viewing and correcting all errors is impractical. The parameter display in the Review GUI was designed to facilitate reviewing, by allowing simultaneous visualization of all frames using an approach adopted from spike sorting software . Namely, in spike sorting software, multidimensional spike shape data is often represented by a small number of principal components. Principal component analysis is not easily applicable to variable movie images, but their dimensionality can be effectively reduced by representing their salient features. The advantage of such a display is that it allows viewing, marking, or changing settings for frames that share common attributes. In particular, the ability to focus on rapid changes and discrepancies between different settings is useful for identifying frames with likely errors. The ability to target on specific arena locations (Fig. 12b–d) allows focusing on those frames where correct detection is critical. Practically, the use of keyboard shortcuts can significantly accelerate the reviewing and correction process.
In summary, OptiMouse is a fully documented MATLAB-based open source program, which users can expand according to their own particular needs. We anticipate that OptiMouse will be very useful to other researchers in the field, providing a simple, accurate, and free solutions for analysis of mouse position data.
I thank Oksana Cohen for extensively and patiently testing the OptiMouse code, for providing many useful suggestions, and for collecting most of the data presented in this manuscript. I thank Ankur Gupta for helpful discussions and insights during initial stages of development and Dorrit Inbar for many helpful discussions on the code. I thank Daniel David, Jennifer Spehr, Noa Rosenthal, and Racheli Nisim for testing the code. I thank the labs of Marc Spehr, Rafi Haddad, Josh Goldberg, Yoni Kupchik, and Adi Mizrahi for providing movies for testing, some of which are shown in the manuscript and the manual. I thank members of the Ben-Shaul lab for commenting on the user manual. I thank Dr. Gillian Kay for proofreading and helpful editing suggestions.
This work was funded by the Israeli Science Foundation (grant #1703/16).
Availability of data and material
All source code can be downloaded from the GitHub repository https://github.com/yorambenshaul/optimouse. The GitHub repository also contains the user manual, which is also available as Additional file 1. Video data analyzed in Fig. 18 is included in Additional files 2, 3, and 4. Additional file 1 is a pdf file containing a detailed user manual. Additional files 2, 3, and 4 are mp4 videos for the data whose analysis is described in the Results section and in Fig. 18. Each of these movies contains images of three simultaneously recorded arenas, of which only one was used in each example in Fig. 18. The left arena in Additional file 2 corresponds to movie 1. The center arena in Additional file 3 corresponds to movie 2. The left arena in Additional file 4 corresponds to movie 3.
YB-S wrote the code and analyzed the data described here.
I declare that I have no competing interests associated with this publication.
Consent for publication
Ethics approval and consent to participate
All experiments from which videos were taken were approved by the relevant ethical committees of the research institutes.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- da Lin Y, Zhang SZ, Block E, Katz LC. Encoding social signals in the mouse main olfactory bulb. Nature. 2005;434(7032):470–7.View ArticlePubMedGoogle Scholar
- Cichy A, Ackels T, Tsitoura C, Kahan A, Gronloh N, Sochtig M, et al. Extracellular pH regulates excitability of vomeronasal sensory neurons. J Neurosci. 2015;35(9):4025–39.View ArticlePubMedGoogle Scholar
- Root CM, Denny CA, Hen R, Axel R. The participation of cortical amygdala in innate, odour-driven behaviour. Nature. 2014;515(7526):269–73.View ArticlePubMedPubMed CentralGoogle Scholar
- Ferrero DM, Lemon JK, Fluegge D, Pashkovski SL, Korzan WJ, Datta SR, et al. Detection and avoidance of a carnivore odor by prey. Proc Natl Acad Sci U S A. 2011;108(27):11235–40.View ArticlePubMedPubMed CentralGoogle Scholar
- Brock O, Bakker J, Baum MJ. Assessment of urinary pheromone discrimination, partner preference, and mating behaviors in female mice. Methods Mol Biol. 2013;1068:319–29.View ArticlePubMedGoogle Scholar
- Roberts SA, Davidson AJ, McLean L, Beynon RJ, Hurst JL. Pheromonal induction of spatial learning in mice. Science. 2012;338(6113):1462–5.View ArticlePubMedGoogle Scholar
- Holly KS, Orndorff CO, Murray TA. MATSAP: an automated analysis of stretch-attend posture in rodent behavioral experiments. Sci Rep. 2016;6:31286.View ArticlePubMedPubMed CentralGoogle Scholar
- Reeves SL, Fleming KE, Zhang L, Scimemi A. M-Track: a new software for automated detection of grooming trajectories in mice. PLoS Comput Biol. 2016;12(9):e1005115.View ArticlePubMedPubMed CentralGoogle Scholar
- Gomez-Marin A, Partoune N, Stephens GJ, Louis M, Brembs B. Automated tracking of animal posture and movement during exploration and sensory orientation behaviors. PLoS One. 2012;7(8):e41642.View ArticlePubMedPubMed CentralGoogle Scholar
- Zarringhalam K, Ka M, Kook YH, Terranova JI, Suh Y, King OD, et al. An open system for automatic home-cage behavioral analysis and its application to male and female mouse models of Huntington's disease. Behav Brain Res. 2012;229(1):216–25.View ArticlePubMedGoogle Scholar
- Dagan SY, Tsoory MM, Fainzilber M, Panayotis N. COLORcation: a new application to phenotype exploratory behavior models of anxiety in mice. J Neurosci Methods. 2016;270:9–16.View ArticlePubMedGoogle Scholar
- Perez-Escudero A, Vicente-Page J, Hinz RC, Arganda S, de Polavieja GG. idTracker: tracking individuals in a group by automatic identification of unmarked animals. Nat Methods. 2014;11(7):743–8.View ArticlePubMedGoogle Scholar
- Ohayon S, Avni O, Taylor AL, Perona P, Roian Egnor SE. Automated multi-day tracking of marked mice for the analysis of social behaviour. J Neurosci Methods. 2013;219(1):10–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Wiltschko AB, Johnson MJ, Iurilli G, Peterson RE, Katon JM, Pashkovski SL, et al. Mapping sub-second structure in mouse behavior. Neuron. 2015;88(6):1121–35.View ArticlePubMedPubMed CentralGoogle Scholar
- Hong W, Kennedy A, Burgos-Artizzu XP, Zelikowsky M, Navonne SG, Perona P, et al. Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning. Proc Natl Acad Sci U S A. 2015;112(38):E5351–60.View ArticlePubMedPubMed CentralGoogle Scholar
- Puscian A, Leski S, Kasprowicz G, Winiarski M, Borowska J, Nikolaev T, et al. Eco-HAB as a fully automated and ecologically relevant assessment of social impairments in mouse models of autism. eLife. 2016;5.Google Scholar
- Crawley JN. What’s Wrong with my Mouse? Behavioral Phenotyping of Transgenic and Knockout Mice. 2nd ed. Hoboken: Wiley-Interscience; 2007. p. xvi–523.View ArticleGoogle Scholar
- Hazan L, Zugaro M, Buzsaki G. Klusters, NeuroScope, NDManager: a free software suite for neurophysiological data processing and visualization. J Neurosci Methods. 2006;155(2):207–16.View ArticlePubMedGoogle Scholar