There are many free and open source image and video annotation tools that can be used with Annolid. Here we describe one recommended Python tool, labelme, by Kentaro Wada.
The overall workflow is to first take a video file and extract a number of frames from it as JPEG files, using Annolid's extraction tool or another tool or service. Once you have a directory (folder) full of JPEGs, follow the instructions on this page to (1) label each frame using Labelme, producing a .json metadata file for each frame, and (2) convert a set of these labels into COCO format for use in training the Annolid network.
The first option is from the pre-build binaries release section.
Or if you want to develop and modify the source code, please navigate to your projects directory, or wherever you intend to install labelme.
git clone https://github.com/wkentaro/labelme.git
cd labelme
pip install -e .For more information, please consult the installation instructions in the Labelme Github repo.
Create a labels.txt file that contains all the annotation labels that you intend to use on a given set of video files. This file can live wherever is most convenient for you; one good choice is in the directory that contains all of the (similar) videos that you intend to label, all of which presumably will be labeled with the same set of labels. An example of such a file is:
__ignore__
_background_
mouse
body
body_centroid
left_ear
right_ear
nose
head
hand
arm
wall
corner
base_of_tail
grooming
rearing
object_investigation
Using the Annolid frame extraction tool, or another selective frame extraction tool, is recommended. But, if you prefer, you also can extract all of the frames from a video my_video.mp4 as JPEG files using a labelme tool.
cd labelme`
python video_to_images my_video.mp4
Whatever technique you use, you should end up with a directory containing all extracted frames as JPEGs. Generally this directory should bear the same name as the video file from which the frames were extracted (here: my_video)
For a folder named my_video, full of JPEGs, and a labels file named labels.txt:
cd labelme
labelme my_video --labels labels.txt
This will start up the labelme GUI, enabling you to go through each JPEG frame in turn and label the relevant features within each frame according to your goals. You will see the list of labels from labels.txt in the labelme GUI, along with the first frame in the directory.
For information on just how to use labelme to label frames, please consult the instructions and tutorials on the Labelme Github page, and the provided video annotation examples.
Saving the annotations made on each frame will produce a corresponding .json metadata file -- one for each JPEG -- in the same directory as the JPEGs themselves.
This labelme tool can be used to quickly check the labeling of single .json files.
cd labelme
labelme_json_to_dataset xxxxxxxxx.json
After labeling is complete, this tool will take all of the JPEG frames, .json metadata, and label.txt information in a designated directory (here: input_dir) and convert these data into COCO format in a different directory (output_dir). (The COCO-formatted data in output_dir then will be used by Annolid for training instance segmentation models).
cd labelme
python examples/instance_segmentation/labelme2coco.py --labels labels.txt input_dir output_dirwhere input_dir might be my_video and output_dir might be my_video_coco.
NOTE: If you encounter problems running labelme2coco.py on Windows or installing its dependencies, see Appendix 1 below for a solution.
When using Anaconda under Windows, running labelme2coco.py for the first time will request that the pycocotools dependency be installed. As of this writing, there is no conda package for pycocotools, so PIP must be used:
pip install pycocotools
The package and several dependencies should download fine; however, installation will fail if there is no C++ compiler available. Specifically, the installer will request Microsoft Visual C++ 14.0. To acquire and install this, first download VisualStudio. The free community version is fine (as of this writing, Visual Studio 2019 is current, and works fine for present purposes). Once the installer is running, select "Desktop development with C++" as shown below for installation. Nothing else is necessary (though even this will take up about 2 GB of drive space). Once this is installed, and the computer rebooted, you should be able to install pycocotools and then run labelme2coco.py.
If you have more complicated problems installing pycocotools under Windows, see this discussion.
Click Edit -> Keep Previous Annotation
D : Next Image
A : Prev Image
Labelling a person in an image: Use a polygon to outline the person's body. Labelling a car in an image: Use a polygon to outline the car. Labelling a road in an image: Use a polygon to outline the road. Labelling a pedestrian crossing in an image: Use a polygon to outline the pedestrian crossing. Pitfalls to avoid when labelling:
Avoid using too few points to define a polygon. For example, using only two points to define a polygon will result in an invalid shape. Avoid using too many points to define a polygon. This can make the label more difficult to understand and can also increase the file size of the labelled image. Be careful when creating a rectangle label. Make sure to click and drag the mouse to create the rectangle, rather than just clicking to create individual points. It's also important to ensure that the labels are accurate and consistent. For example, if you are labelling objects in an image, make sure to use the same label type (e.g., polygon) for all objects and to label all relevant parts of the object (e.g., the entire body of a person, rather than just their head).