RPA platforms use IDs to identify the elements inside the target applications and then perform actions like mouse clicks or sending Hotkeys to get the desired end result. When we work with remote systems and databases, like Citrix or the open web, it is not easy for the robot to identify the selector regions on screen.
In case of Citrix automation/automation in virtual environments, there is no way to get the required information for automation such as the element ID. The information is available only as a pixel image and apparently what the bot sees is an image. Image recognition is the way forward when you are faced with this situation. It looks easy but it has its own challenges like colors might vary, resolution issues which can cause failure in image recognition etc.
Is there a way to retrieve the desired text data from the applications?
The answer is Yes, OCR engine is the Hero for image based automations, which can magically recognize the text out of an image.
But does it ensure 100% accuracy in the output?
No, OCR engines currently available in the market are not 100% perfect. Hence, in some cases there are chances that our applications go for a toss if we consider the output given by OCR engines.
Is there a workaround?
Yes! The smartest and easiest way is to retrieve either Partial (in case of structured data having known what part of data needs to be fetched) or complete page of an application which can be extended to multiple pages depending upon the application from which data has to be fetched.
My BOT has to open a remote desktop, login to a jpgage and get me the data from the 3rd column of the first row, of a structured table, which is a part of the jpgage.
First thing that anyone would think of is, find an image of the table name and fetch the relative positional data by specifying the co-ordinates to be fetched. That’s quite simple!
But here comes the challenging part!
The cells of this table are not of fixed co-ordinates. The cell shrinks when there is less or no data and expands to wrap-up the data when the data in the cell is big. So, the row size as well as the column size changes and my relative positional fetching of data will fail miserably!
The simple yet smart approach to solve this issue is –
The scanned PDFs are treated as data that cannot be readable. The solution works in all the applications where copying the data is allowed.
Image recognition, when paired with other RPA features can expand the capabilities of business process automation. If you are looking for an automation tool which works for your process, Nalashaa can assess your processes, work out an automation score and the overall predicted ROI.
Vinutha is a passionate developer who tends to get lost in her work. She enjoys interacting with clients and is excellent at developing a good rapport with them. She is a dreamer, and loves everything conducive to getting lost in thought - from long drives to walk in the beaches.