A tap tied to fixed coordinates assumes that the target will always be in the same place. That assumption breaks when a list scrolls, a banner changes the layout, or the same screen renders differently on another device. If the target has a stable label, it is often better to find the words and tap where they appear.
EmuloMobile includes a Tap text action for that job. It reads the screen with OCR on the Android device, filters the recognized results by confidence, and taps the center of the first matching line. You can restrict the reading to a region so the search is faster and less ambiguous.
Use automation only where the app or service allows it. This tutorial changes how a target is located; it does not bypass third-party rules or confirmations.
When tapping text is better than tapping a point
Use text as the target when the label remains meaningful but its position can move. Common examples are a Continue button below variable content, an item in a scrolling list, or an option that shifts after a notice appears.
A coordinate tap can still be the simpler choice for a fixed control with no readable label. An image click is a better fit when the target is an icon or when typography makes OCR unreliable. The practical rule is to use the most stable signal available:
- use Tap text when the written label identifies the target;
- use Image click when the appearance identifies it;
- use Tap (x, y) when the position is intentionally fixed.
If you need a broader comparison of visual signals, see the guide to on-device Android automation.
Before creating the OCR action
Open the target app and leave it on a screen where the label is visible. In EmuloMobile, confirm two settings before testing:
- In Settings, enable the EmuloMobile accessibility service. It performs only the taps and gestures you configure and draws the control overlay.
- Grant Screen capture. OCR analyzes the pixels from the screen capture locally on the device.
Screen capture, text recognition, and image analysis used by the automation run on the device. The recognized screen content is not sent to Sertta or third parties.
Also keep the device in the same resolution and orientation used when the automation was created. EmuloMobile treats those properties as part of the macro and checks them before editing or execution.
How to create a Tap text action in EmuloMobile
Start with an existing automation or select Create automation, name it, and confirm the detected resolution and usage mode. Then follow this sequence in the builder.
1. Add the action
Select Add Action, choose New Action, and then choose Tap text. The action editor opens with the OCR controls for this step.
In Text to find on screen, enter the label as it is displayed. For a button labeled “Continue,” enter Continue; avoid surrounding words that are not part of the target line.
2. Choose partial or exact matching
Accept partial match allows the recognized line to contain the text you entered. It is useful when the screen shows “Continue to payment” but Continue is the stable part.
Require exact text only accepts a normalized line equal to the expected text. Use it when short labels repeat or when tapping a line with extra words would be unsafe.
The comparison is normalized by the app before matching, but OCR still depends on readable pixels. Treat matching mode as a way to control ambiguity, not as a substitute for a clear target.
3. Limit OCR with a region
Select Region and draw a rectangle over the part of the screen where the target may appear. The live overlay is placed over the actual target app, so you can define the area in context.
Do not crop the region to the current letters too tightly. Leave room for the label to move within its expected container. At the same time, avoid including unrelated menus, repeated labels, or a second button with similar text.
A useful region might cover the lower third of a dialog rather than the whole screen. During execution, EmuloMobile reads only that area and translates the recognized text position back to the correct screen coordinates before tapping.
4. Tune minimum confidence only when needed
The action starts with a 50% minimum confidence. Keep that value for the first test. If the text is clearly visible but not found, check spelling and region before lowering confidence. A lower threshold can admit poor OCR results; a higher one can reject stylized or small text.
Change one variable at a time:
- verify the expected text;
- confirm that the region contains the complete label;
- compare partial and exact matching;
- only then adjust minimum confidence.
5. Save explicitly
Return from the action editor and select Save in the builder. Changes remain pending until you save them. Leaving with unsaved changes prompts you to save or discard them.
Practical example: tap a button that moves
Imagine a confirmation screen where a short warning sometimes appears above the Continue button. The button moves down when the warning exists, so a fixed coordinate can hit the warning or empty space.
Configure the action like this:
- Text to find on screen:
Continue - Matching: Require exact text
- Region: the dialog body and footer, excluding the navigation bar
- Minimum confidence: start at 50%
The action does not reuse the coordinate from the setup screen. On every execution, it captures the current screen, runs OCR in the selected region, finds the first recognized line that matches, and taps its current center.
If the same word appears twice inside the region, narrow the rectangle so only the intended button remains. The action deliberately uses the first matching OCR line; the region is therefore the main tool for making the target unambiguous.
Add a condition when the action needs a guard
Locating the target by text solves movement, but it does not define whether the entire step is appropriate in the current state. You can add a condition to the same step when another signal should protect the action.
For example, require the dialog title to appear before tapping Continue. A Text on Screen condition can check whether text contains, equals, differs from, or matches a regular expression. Image and color conditions are also available when they represent the state more reliably.
The condition is evaluated before the action. If it is not satisfied, the tap does not run. Keep the guard distinct from the target: the condition answers “is this the correct screen?”, while Tap text answers “where is the control now?”
Test the step on the real screen
Before running the full automation, place the target app on the expected screen and use Test for the step. EmuloMobile reports whether the conditions passed and whether the action fired.
Check these failure patterns:
- Text not found: confirm the label, search region, and minimum confidence.
- Wrong matching line: use exact matching or narrow the region.
- Tap is correct only in one orientation: return to the resolution and orientation stored by the macro.
- Capture unavailable: stop any active automation and prepare screen capture again.
After the step works alone, test a full pass. Save the automation, return to the automation list, prepare execution, and use the floating control to start, pause, or stop it while watching the target app.
Make the OCR target specific, not fragile
A good OCR action is specific enough to identify one control but flexible enough to tolerate normal movement. Prefer a stable label, choose the strictest useful matching mode, and restrict the search to the smallest practical region. Those three decisions usually matter more than repeatedly changing the confidence slider.
For flows that need memory after the tap, continue with the guide to variables, counters, and conditional actions.
Install EmuloMobile and build a text-aware Android automation directly on the device.