LinkedAI

Flip is an open-source library that allows you to generate synthetic images in a few steps from a small set of images made up of backgrounds and objects (images that would be on the background). To do this, it allows the user to apply transformations on objects such as flipping about an axis, resizing, or rotating. With the above defined, a composition is made with a set of options to establish the final state of the background and the location of the objects. Finally, the user can generate jpg, json, csv and pascal voc files already labeled.

Now let’s see how it works in the practice. In this link you can see an example of how to use the library to create new synthetic images for each background, you can also determine the number of objects to add.

For this example the following images were used (which you can find in the example repository):

‍

‍

Object: penguin in png format with transparent background.
Background: landscape in jpg format.

Some lines of code are explained below.

‍

♦ Flip: allows you to rotate the object or the background about the x, y axis or random.


flip.transformers.data_augmentation.Flip (mode=’random’

‍

In images 1 and 2 a flip was applied to the background and in 3 and 4 it was applied to the object. In 1 and 3 it was with respect to the x axis, while in 2 and 4 it was with respect to the y axis.

‍

♦ RandomResize: change the size of the object with respect to itself or the background. The changes can be asymmetric or symmetric with respect to width or height, with the possibility of defining maximum and minimum values.

‍


flip.transformers.data.augmentation.RandomResize    
mode='symmetric_w',   
relation='parent',    
w_percentage_min=0.2,   
w_percentage_max=0.7

‍

‍

♦ Rotate: rotate the objects at angles of 90º, 180º or randomly (between 0º and 360º). The last one allows determining the minimum and maximum values to make the rotations.


flip.transformers.data_augmentation.Rotate mode=’random’, min=45, max=180)

‍

In image 1 the random setup was used, in image 2 theupsidedown or 180º setup , and in image 3 the 90º setup.

‍

After defining the transformations of the objects, a composition is made between them and the backgrounds (to which it’s also possible to apply transformations such as Flip). The way in which the objects will be placed on the backgrounds can be in two ways: by percentage or completely random. The user can determine the minimum and maximum values (for both width and height) where the objects could be positioned.

‍


flip.transformers.domain.randomization.Objects.Random.Position    
x_min=0.3,        
y_min=0.3,     
x_max=0.7,    
y_max=0.7,           
mode='percentage'),
flip transformers.domain randomization Draw

‍

‍

Finally, the user can save the generated images in jpg format.

‍


flip.transformers.io.SaveImage.(OUTPUT_DIR,name)

‍

You can also download the generated labels adding the following transformers to the compose object.

‍


flip.transformers.labeler.CreateBoundingBoxes(),
flip.transformers.io.CreateJson(OUTPUT_DIR,name)

‍

The first line will generate the bounding boxes around the targeted object and the second one will save them in a JSON format, you can also save the labels in csv and pascal voc formats. The .json file contains an array of dictionaries (one per image generated) with two keys: name and pos.

‍

name: contains the name of the image.
pos: contains the position of the objects in the background. The values x and y determine the top-left coordinates and the values w and h are the width and height of the bounding box (all units are in pixels).

For this example, all images have the same name as it was used only one image.

‍


[{“name”: “pinguino.png”,
“pos”: {“x”: 200, “y”: 439, “w”: 112, “h”: 202}},
{“name”: “pinguino.png”, 
“pos”: {“x”: 126, “y”: 449, “w”: 112, “h”: 245}}, 
{“name”: “pinguino.png”, 
“pos”: {“x”: 345, “y”: 436, “w”: 112, “h”: 234}}, 
{“name”: “pinguino.png”, 
“pos”: {“x”: 621, “y”: 384, “w”: 112, “h”: 239}}]

‍

The csv format will look as follows.

‍

pinguino.png 200 439 112 202
pinguino.png 126 449 112 245
pinguino.png 345 436 112 234
pinguino.png 621 384 112 239

‍

Pascal voc format will generate a .xml file with additional information, the name of the image, its position, dimensions (width, height, and channels), the name of the images used as objects, and the values of the top-left coordinates (xmin, ymin) and bottom left (xmax, ymax) of each bounding box. You can see an example below.

<annotation>
<folder>synthetic_images</folder>
<filename>derevya-dorogi-pejzazh-sneg-zima-26355__45763af5-a813–4f37–87d1–0bc78502fc0f.jpg</filename>
<path>synthetic_images/derevya-dorogi-pejzazh-sneg-zima-26355__45763af5-a813–4f37–87d1–0bc78502fc0f.jpg</path>
<source>
<database>Unknown</database>
</source>
<size>
<width>1120</width>
<height>700</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
<object>
<name>pinguino.png</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>200</xmin>
<ymin>439</ymin>
<xmax>312</xmax>
<ymax>641</ymax>
</bndbox>
</object>
<object>
<name>pinguino.png</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>126</xmin>
<ymin>449</ymin>
<xmax>238</xmax>
<ymax>694</ymax>
</bndbox>
</object>
<object>
<name>pinguino.png</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>345</xmin>
<ymin>436</ymin>
<xmax>457</xmax>
<ymax>670</ymax>
</bndbox>
</object>
<object>
<name>pinguino.png</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>621</xmin>
<ymin>384</ymin>
<xmax>733</xmax>
<ymax>623</ymax>
</bndbox>
</object>
</annotation>

‍

To visualize the generated bounding boxes, you have to generate and save the JSON format first and then you can use the following notebook: show_labels.ipynb (you can find it in the examples of FLIP library). Bounding boxes will look like this:

‍

‍

Creating Synthetic Images with Flip