Easy image recognition

Best practices, code snippets for common functionality, examples, and guidelines.
Posts: 87
Joined: Sun Aug 05, 2012 11:59 am
Location: Bilbao, Spain

Easy image recognition

Post by mdgairaud » Thu Mar 20, 2014 9:22 am

Some people asked how I made the captcha image recognition [http://www.ranorex.com/forum/reading-captcha-t3629.html] and if I can post it in forum.

Well, first of all I must say that this code only works with captchas that have regular character, if characters are randomly stretched or deformed isn't work. And of course, this method is made ad-hoc for some of our test, so it might never fit with your needs.

I take the code I used and adapt a little with some commentaries inside. Any question or improves are welcome :D :D

Code: Select all

     This method checks some pixels ins order to recognize a letter/number. That's the procedure:
     * 1. Capture the image and crop it to fit (with provided coordinates). This step is important to remove empty spaces around the character.
     * 2. Convert to Black and white
     * 3. Check some coordinates inside the character to see if all are Black. More coordinates, more accuracy  :)
     First of all you need to pass an image of the character you want to recognize.    
    CompressedImage miImage = RanorexElement.CaptureCompressedImage();
    Bitmap src = miImage;

    //This rectangle contains the coordinates to crop the image, make sure that the character fits inside this rectangle
    private Rectangle recImagen = new Rectangle(coordenadasImagen[0], coordenadasImagen[1], coordenadasImagen[2], coordenadasImagen[3]);

    private bool Compare(Bitmap miImagen)
        //Crop the image to remove empty spaces besides the letter/number. 
        Rectangle cropRect = recImagen; //recImagen is a Rectangle that fits over number/letter
        Bitmap src = miImagen;
        Bitmap target = new Bitmap(cropRect.Width, cropRect.Height);

        using (Graphics g = Graphics.FromImage(target))
            g.DrawImage(src, new Rectangle(0, 0, target.Width, target.Height),

        //Convert image to B/W
        target = CambiarColor(target);
        //Check some points of image to recognize it. Coordinates to check are stored in '_coordenadasComprobacion' like: xxyy,xxyy,xxyy...
        //To check that a point is correct, the sum of the colors R, G, B must be 0. That's why image is converted to B/W
        int sumaColor = 0, parX, parY;
        foreach (string par in _coordenadasComprobacion)
            parX = int.Parse(par[0].ToString());
            parY = int.Parse(par[2].ToString());
            sumaColor +=
            target.GetPixel(parX, parY).R +
            target.GetPixel(parX, parY).G +
            target.GetPixel(parX, parY).B;

        //If the sum of all colors is 0, the number/letter is correct (true) or incorrect (false)
        if (sumaColor == 0)
        { return true; }
        { return false; }


    //This function calculates the sum of the three colors of each pixel and depending the threshold, it's converted to black and white
    private int _umbral = 200; // this variable sets threshold. Range is from 0 to 765 (The sum of R(255), G(255) and B(255))
    private Bitmap CambiarColor(Bitmap miBMP)
        Color miColor = new Color(); int prom;
        for (int y = 0; y < miBMP.Height; y++)
            for (int x = 0; x < miBMP.Width; x++)
                miColor = miBMP.GetPixel(x, y);

                //take pixel color sum
                prom = (miColor.R + miColor.G + miColor.B);

                //check if it's over or below the threshold
                if (prom > _umbral)
                    miBMP.SetPixel(x, y, Color.White);
                    miBMP.SetPixel(x, y, Color.Black);
        return miBMP;

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA

Re: Easy image recognition

Post by Support Team » Thu Mar 20, 2014 3:04 pm

Hi mdgairaud,

Thank you for providing your experiences to the Ranorex community. We really appreciate that :)