Traversal of image gallery using Hand Gestures

c++, computer vision, linux, opencv, webcam

This project aims to slide images in an image gallery using handgestures. It is built with the help of opencv and c++ using a linux environment and a webcam. The source code can be found here.

It is based on Simen Andresen’s blogpost. The hand is detected with the help of color sampling. The hand is then extracted from the background by using a threshold using the sampled color profile. Each color in the profile produces a binary image which in turn are all summed together. A nonlinear median filter is then applied to get a smooth and noise free binary representation of the hand. After that, the convexity defects are removed. The properties determining whether a convexity defect is to be dismissed is the angle between the lines going from the defect to the neighbouring convex polygon vertices.

The traversal through image gallery is done by taking a folder with image suffix of jpg and incremental filenames for simplicity. Since the analysis is done frame by frame, therefore to minimise error in detecting a hand contour, a reference range is taken in the middle of the video screen. When the hand crosses a particular distance from the center of the screen(140 pixels in my case), it detects movement and hence increments/decrements the filename of the images depending upon the relative vector direction. Below is the attached code snippet which implements the sliding of the images.

code snippet
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
if(countFrame==1)
	{
	 ref = ((hg->bRect.tl()+hg->bRect.br())* 0.5);
	 countFrame++;
	}


    if(firtTime)
    {
    	stringstream ss;
    	  ss << setw(2) << setfill('0') << counter; // 00, 01, 02, etc...
    string number = ss.str();
    string name = folder + number + suffix;
    	myImage = imread(name);

        if(!myImage.empty())
    {
	    imshow("HEYO", myImage);
    }
        firtTime=false;
    }

    if((center.x - ((hg->bRect.tl()+hg->bRect.br())* 0.5).x<5  && center.x - ((hg->bRect.tl()+hg->bRect.br())* 0.5).x> -5) && swipeCount<1)
	{
		swipeCount++;
	}

	if((center.x - ((hg->bRect.tl()+hg->bRect.br())* 0.5).x >140) && counter>0 && swipeCount>0)

	{

	int xpos=(m->src).cols/2;
	float fontSize=0.7f;
	Scalar fColor(245,200,200);
	int ypos=(m->src).rows/2;
	string info=string("Left Window slide accomplished ");
	 printText(m->src,info);
	 // ref = ((hg->bRect.tl()+hg->bRect.br())* 0.5);
	     counter--;
	     swipeCount--;

	 stringstream ss;

	   ss << setw(2) << setfill('0') << counter; // 00, 01, 02, etc...
    string number = ss.str();
    string name = folder + number + suffix;

	 myImage = imread(name);

	 if(!myImage.empty())
    {
    imshow("HEYO", myImage);
    }


	}

	

	if((center.x - ((hg->bRect.tl()+hg->bRect.br())* 0.5).x < -140) && counter<5 && swipeCount>0)

	{
	int xpos=(m->src).cols/2;
	float fontSize=0.7f;
	Scalar fColor(245,200,200);
	int ypos=(m->src).rows/2;
	string info=string("Right Window slide accomplished ");
	 printText(m->src,info);
	 // ref = ((hg->bRect.tl()+hg->bRect.br())* 0.5);

	     counter++;
	     swipeCount--;
	     stringstream ss;

	       ss << setw(2) << setfill('0') << counter; // 00, 01, 02, etc...
    string number = ss.str();
    string name = folder + number + suffix;


	myImage = imread(name);

	if(!myImage.empty())
    {
	    imshow("HEYO", myImage);
    }


	}
Share Comments