Table of Contents
Desktop Adaptor
Introduction
The new desktop adapter automates the use of client applications on Windows and Linux PCs. That is, with this adapter we can open any application we have on our desktop.
Initialization parameters
- Default Screen Number: it will indicate the monitor on which it will be executed: it will take the value 0 as the default value.
- Images Repository Path: it will indicate the path (or part of it) of the folder where all the images that we have previously captured will be.
Predefined parameters (PF):
- clearCtrlADel: deletes the content or value of an element on which it is positioned (field or zone of an application). Selects the content with Ctrl+A and the deletes. This function is used when the selection is made with Ctrl+A. It receives an optional input parameter which is the delay in millisecond between keys.
- clearCtrlEDel: deletes the content or value of an element on which it is positioned (field or zone of an application). Selects the content with Ctrl+E and the deletes. This function is used when the selection is made with Ctrl+E. It receives an optional input parameter which is the delay in millisecond between keys.
- click: click on the element that matches the image that has been passed in the entry. The images where we want to click, we will have previously captured and saved them and indicated the path where they are saved.
- clickNElement: it searches the screen, which at that moment is being displayed, for the elements that coincide with the image that has been passed in the input and clicks on the element that occupies the position (1, 2, 3, … N) that has been passed as input in a field of integer type. This function always goes through the elements found from left to right and from top to bottom. This function is used when there are several equal images aligned in rows and columns.
- clickCoordinates: click on the specific point defined by the X and Y coordinates received as input parameters.
- countElements: searches the currently displayed screen for items that match the image passed in the input and returns the number of items found in an integer variable.
- doubleClick: double clicks the mouse on the element that matches or agrees with the image that has been passed in the input.
- doubleClickCoordinates: double clicks the mouse on the specific point defined by the X and Y coordinates received as input parameters.
- getTextFromImage: gets the text from the image indicated in the input parameter, using the language of the received text also as input. Optionally, a region of the image can be used. If the optional parameters of the region have been informed, then save the screenshot of the region as evidence.
- getTextFromRegion: gets the text from the region specified by the input parameters, using the language of the received text as input as well. Save the screenshot of the region as evidence.
- isExist: checks if there is any element on the currently displayed screen that matches the image passed in the input. Returns true if any element has been found and false if none has been found.
- maxWindow (): will maximise the application window currently in focus, no parameters required. Returns true or false depending on whether it succeeds in executing the FP.
- openApp: opens an application by running the executable passed in the input parameter.
- paste: pastes the received text at the current cursor position.
- pasteCoordinates: pastes the text received in input at the specific point defined by the X and Y coordinates also received as input parameters.
- rightClick: clicks with the right mouse button on the element that matches or agrees with the image that has been passed in the entry.
- rightClickCoordinates: clicks with the right mouse button on the specific point defined by the X and Y coordinates received as input parameters.
- sendKeysCombination: sends a combination of keys to execute where the cursor is at that moment. The keys to send are separated by +, e.g. 17+69+162. This function has three input parameters:
- “Key+Key”: is mandatory and is the keys to send.
- “Delay milliseconds”: parameter is optional and is the delay in milliseconds between keys.
- “Repeat”: is also optional and is the number of times (number of repetitions) that the sending of these keys must be executed, if no default value is set, it considers the value 1.
- takeScreenshot: takes a screenshot of the screen that has the focus. The screen that has the focus is the one the user is currently on.
- takeScreenshotRegion: takes a screenshot of the region specified by the input parameters.
- type: writes the received text at the current cursor position.
- typeCoordinates: it writes the text received at the specific point defined by the X and Y coordinates received also as input parameters.
To know the coordinates corresponding to a point on the screen, Cursor Guide has been used, an App that can be downloaded at: https://sourceforge.net/projects/cursorguide/
To know the corresponding value of each key on the keyboard, Keyboard State View has been used, a program that can be downloaded at:
https://www.nirsoft.net/utils/keyboard_state_view.html“
For using the text management functions of the Desktop Adaptor, dictionaries are needed, (depending on the language of the text), which can be downloaded from the page:
https://github.com/tesseract-ocr/tessdata
These files (TRAINEDDATA extension) must be in the path: C:\AppTast\tessdata
For German, English and Spanish there would be the following files:
