Desktop picker

Desktop picker is a tool that allows you to specify the elements of desktop or application interface that the bot needs to interact with.

There are three ways to open the Desktop picker:

1. The "Pick New Element" button in the activity options, such as Click on element in the "GUI Automation" → "Mouse" section. Clicking this button will open the selection mode, where you can specify the required element. After picking the element, the Desktop Picker window appears: there you can test and modify the picked element.
2. Tools → "Process Recorder" → "Desktop". In this case you will immediately go into item selection mode. At a time you can select up to ten elements (for click or input), and during the selection the Desktop picker window will not be displayed. After the selection you will return to the Studio Pro window. The corresponding activity blocks (Click on element or Input to desktop app) will appear in the canvas.
3. Elements toolbar. Use this toolbar to edit an element or create a new one. To do this you should press the "Change the element" button in the element. This will open a selector window, where you can select a new desktop element using the "Replace element" button.

Working with the tool is quite easy. When you click the "Pick New Element" button, Studio Pro minimizes and starts a selection mode where you can pick the desired interface element. When you hover the cursor over an element, it will be surrounded by a red frame. Pressing the CTRL+X key combination marks the selected item. The ESC button restarts the element selection mode.

After the item is selected, the Desktop picker window opens.

To the right of the item name its status is displayed, showing whether the item is found, not found or multiple items found.

Advanced options

The advanced options represent the structure (selector) of the picked element, which usually consists of tabs, and each tab consists of attributes and their values. This displays a hierarchy of how the interface element was created by the application developers. You can change, select or exclude attributes to achieve greater accuracy and stability in your interaction with the element.

Take a look at an example of a selector on the Notepad application.

Tabs

On the left side of this window there are two tabs - "Window", indicating the application window itself, and "Edit", indicating the selected input field in Notepad.

Note the checkboxes to the right of the tab names. These checkboxes allow you to exclude tabs from the selector. In some situations, the number of tabs can be quite large, which can make the selector less stable. At the same time, these tabs are not always necessary for the unambiguous definition of the item (as a rule, we mean "intermediate" tabs - tabs between the first and the last one). In this case, excluding these tabs from the selector may make the selector more stable and at the same time less cumbersome. However, keep in mind that excluding any tabs from the selector in some cases may cause the Desktop Picker to run slower. 

"Name", "Calculation", "Value"

Each tab contains columns "Name", "Calculation", and "Value". Let's discuss the meaning of these columns using one of the attributes of the selected notepad element as an example (see above). Let it be the "Name" attribute:

1. The Attribute column is equal to Name - essentially the name of the element from the Window tab.
2. The Calculate column is set to Is. This parameter allows you to specify how you want the selector to search for this attribute. If the value is Equal (as in our example), then the bot will search for the selector whose Name attribute is exactly equal to the value from the table.
3. Column Value equals Notepad, so the Window tab in this case corresponds to the Notepad window itself, which has exactly this title.

Depending on the scenario of working with the selector, the values of some attributes can change dynamically. For example, if you start typing text in Notepad, the title of the window will change - a symbol "*" will appear before the title (which is true - look at the title of the Notepad window itself to see this). The same change will happen to the attribute in the Desktop Picker.

This means that the previous version of the selector (without the "*" in the Name attribute) is not adapted to the case when some text was initially written to notepad, which would lead to a bot error.

To avoid such errors, there are ways to adapt for dynamic attribute changes. These ways are embedded in the " Calculate " column in the table with the selector. Let's take a closer look at them:

1. Is indicates that the value of this attribute must be checked for strict equality when searching for an element. In the Notepad example, this means that the window title must be strictly equal to New Text Document - Notepad. That is, an element that has a Name attribute equal to, say, Text Document - Notepad will no longer be taken into account by the bot.
2. Calculate- allows you to set a variable defined in workflow as an attribute value. When searching for a selector with a computed attribute, equality is also strict (only the attribute value is no longer defined by fixed text, but by a variable).
3. Wildcard allows you to set a character substitution rule in the value string. This allows you to adapt to dynamic attributes. There are two characters with which you can set the wildcard rule:
   1. "*" - replaces any number of characters. For example, if you set the value *Notepad, the bot will search for all items with the Name attribute equal to any character set, followed by a space, a dash, a space and the word Notepad.
   2. "?" - replaces only one character. Thus, if you set ?New Text Document - Notepad, the bot will search for all items that contain any character (one) and then exactly the phrase New Text Document - Notepad.
4. Contains - allows you to specify the part of the text that must be contained in the attribute value. That is, when searching the bot will search for items where the attribute exactly contains the specified text. For example, if the attribute Name is specified in the selector, which "Contains" the value Untitled - Notepad, and the name of the Notepad window is *Untitled - Notepad, then such a selector will be found.

The "Test" button in the selector window allows you to test if an item is correctly detected with the selected selector.

Replace element

To understand how the replace element function works, let's analyze the basic scenario of its application. Suppose you select an element, run the bot - everything works as it should. After making some changes in the application, the bot, for some reason, stops working correctly (or doesn't start at all). Next, you are likely to try selecting the item again, repeating the same actions and launching the bot. But this may not work either. This situation can occur due to dynamic changes in the parameters of the application which the bot interacts with. 

Let's take an example of such situation when working with the standard "Notepad".

If you have just opened "Notepad" and have not yet entered any text, the name of the text file at the top of the window will be as Untitled - Notepad.

If you start typing any text in the input box, the name at the top will change. Now it will look like *Untitled - Notepad.

Let's explore another case: you open an already created text file with some content and the name you set when you saved it.

It becomes clear that at different stages of working with the "Notepad", its name and contents will change. These are the dynamic parameter changes we mentioned above. Let us have a closer look at them.

Let's take the Click on element activity as an example. Let's select any "Notepad" element via the Desktop Picker. For example, you can select the input field or the "File" button. The item will be detected without any problems and the bot will be able to interact with it, you will see the message "Found 1 item". Save the current current item with the "SAVE" button. Run the bot, make sure it works.

However, you may encounter a scenario where the bot needs to work with different text files. Let's go back to "Notepad" and enter the number 123 in the input field, then the title of the "Notepad" window will change - until the file is saved, the "*" symbol will appear before the title. Thus, the entire header will have the look of *Untitled - Notepad. Let's try to run the bot: it won't work and we will see an error message in the console.

Let's consider another scenario. In the current "Notepad" window, open an existing text file. Note the "Notepad" title - it will look like {{Document name}} - Notepad, which means that the title can be anything. In the screenshots below, the document name will look like a Text file.

Let's try running the bot again. In this case the bot will not work either and we will see an error.

Let's analyze what goes wrong and how to fix it.

1. Navigate to the "Edit element" tab in the activity parameters.
2. Press on the "CHANGE THE ELEMENT" button.
3. Test it by pressing the "TEST" button. The element will not be found.

Let's expand the "Advanced щptions" to take a closer look at the parameters of the selected item. Let's go to the "Window" tab, which has a Name attribute showing the title of the "Notepad" window. Since the attribute value is set so that the title must be strictly equal to the Untitled - Notepad value, Desktop Picker cannot detect the file with the changed title.

Thus, because of changes in certain parameters (and they can also change during bot execution), the element is not found and the bot cannot work correctly.

Now let's see how we can easily solve this situation. Let's click the "REPLACE ELEMENT" button in the Desktop Picker window, select the item of the last opened "Notepad" again and press CTRL+X. This will open a window like this:

The top of this menu shows the three available versions of the workflow:

1. Old version. This is the version of the element that was last before you clicked "REPLACE ELEMENT". In our case, it is the version where the Name attribute is Untitled - Notepad.
2. New version. This is the version of the element we just selected. That is, the Name attribute is {{Document name}} - Notepad.
3. Improved version. This version of the element combines the features of the old and new versions to create a selector that can detect the element with any "Notepad" window title.

Select the " Improved" version of the selector and expand the "Advanced options". Here we will see that the changed parameters are highlighted in orange. Now the item will be detected in any case, since the parameters Wildcard→*- Notepad will be selected for the Name attribute, i.e. the platform will search for elements that have in the Name attribute the phrase - Notepad and any number of any characters up to this phrase.

Next, all you have to do is save the element selection in this form. You can also manually adjust the "Name", "Calculation" and "Value" so that the elements you want are defined. We told you more about that earlier in this article.