Pattern API

Definition of Pattern class

The type definition of the pattern object Pattern is as follows, which can be used as a reference for the API list.

interface PatternControl {
    click(x: number, y: number, mouseKey: MouseKey): Promise<void>;
    dblClick(x?: number, y?: number, mousekey?: MouseKey): Promise<void>;
    drag(x: number, y: number): Promise<void>;
    drop(x: number, y: number): Promise<void>;
    exists(seconds: number): Promise<boolean>;
    findAll(): Promise<PatternControl[]>;
    highlight(duration: number): Promise<void>;
    hover(): Promise<void>;
    locate(): Promise<PatternControl>;
    modelProperties(): {[x: string]: any};
    moveMouse(x?: number, y?: number, seconds?: number): Promise<void>;
    rect(): Promise<Rect>;
    score(): Promise<number>;
    takeScreenshot(filePath?: string): Promise<string>;
    wait(seconds: number): Promise<void>;
}

class Pattern():
	def click(x: Optional[int]=None, y: Optional[int]=None, mouseKey: Optional[int]=None) -> None
	def dblClick(x: Optional[int]=None, y: Optional[int]=None, mouseKey: Optional[int]=None) -> None
	def drag(x: int, y: int) -> None
	def drop(x: int, y: int) -> None
	def exists(seconds: Optional[int]=None) -> bool
	def findAll() -> "List[Pattern]"
	def highlight(, duration: Optional[int]=None) -> None
	def hover() -> None
	def locate() -> "Pattern"
	def moveMouse(x: Optional[int]=None, y: Optional[int]=None, seconds: Optional[int]=None) -> None
	def rect() -> "Rect"
	def score() -> int
	def takeScreenshot(filePath: Optional[str]=None) -> str
	def wait(seconds: Optional[int]=None) -> None

Because the pattern object can stand alone or create sub-pattern objects, the Pattern object can be accessed from any library's Model object. For example, it can be accessed from the Model object in leanpro.common if it is used independently, or from the Model object in libraries such as leanpro.win, leanpro.qt or leanpro.java, etc. If it is a sub-pattern, it is obtained from the library corresponding to the parent object.

API Descriptions

click(x?: number, y?: number, mouseKey?: MouseKey): Promise<void>

Click on the target pattern, you can specify the offset of the click, the default click position is the pattern proper center. If the clicked pattern has not been matched to the result, it will be matched before the click.

• x: (Optional) number type, horizontal coordinate, defaults to the horizontal center of the pattern object.
• y: (Optional) number type, vertical coordinate, defaults to the vertical center of the pattern object.
• mouseKey: (Optional) MouseKey type, the key used for the click, default is 1, represents the left mouse button.
• Return value: is an asynchronous method without any return value.

dblClick(x?: number, y?: number, mouseKey?: MouseKey): Promise<void>

Double-click the pattern control with the same parameters as click.

drag(x?: number, y?: number): Promise<void>

Press the left mouse button at the position where the pattern object matches the result, which is the first step of the drag-and-drop operation, and wait for the command to release the mouse. The (x, y) passed in is the coordinate of the click relative to the control, when x and y are both 0 or default click on the center of the control.

• x: (Optional) number type, the horizontal coordinate offset in pixels of the drag starting point relative to the control, left negative right positive. Default is 0, the horizontal center of the pattern object.
• y: (Optional) number type, the horizontal offset pixel of the drag start point relative to the control, left negative and right positive. Default is 0, the vertical center of the pattern object.
• return: asynchronous method that does not return any value.

drop(x?: number, y?: number): Promise<void>

Release the left mouse button at the position where the pattern object matches the result, which is the second step of the drag-and-drop operation. The (x, y) passed in is the coordinate of the click relative to the control, when x and y are both 0 or default click on the center of the control.You can execute the drag() method on the A control and the drop() method on the B control to achieve the effect of dragging and dropping the A control onto the B control. Specific documentation can be viewed in drag and drop method。

• x: (Optional) number type, the horizontal coordinate offset in pixels of the drag start point relative to the control, left negative right positive. Default is 0.
• y: (Optional) number type, the horizontal offset pixel of the drag start point relative to the control, left negative and right positive. Default is 0.
• return value: asynchronous method that does not return any value.

exists(seconds: number): Promise<boolean>

Wait for a certain amount of time and try to match in a loop, if the result is matched within the specified time return ture; more than the waiting time and no result is matched then return false.

• seconds: number type, wait time in seconds.
• return value: Promise<boolean> type, returns true if the result is matched, and false if it times out.

findAll(): Promise<PatternControl[]>

Performs a match on the pattern object and returns all matches, or an empty array if no matches are found.

• Return value: Promise<PatternControl[]> type, asynchronously returns an array of Pattern objects containing all matches, or an empty array if no matches are found.

highlight(duration: number): Promise<void>

Control highlight, you can pass the parameter to specify the highlight duration in milliseconds, the default value is 1000, which means it lasts 1 second.

• duration: (Optional) number type, the duration of the highlight in milliseconds.
• return value: Asynchronous method that does not return any value.

hover(x?: number, y?: number): Promise<void>

Make the mouse cursor hover over the target pattern, you can specify the offset of the hover position, the default position is the pattern proper center. If the pattern being manipulated has not been matched to the result yet, it will be matched before the operation.

• x: (Optional) number type, horizontal coordinate, default is the horizontal center of the pattern object.
• y: (Optional) number type, vertical coordinate, defaults to the vertical center of the pattern object.
• Return Value: Asynchronous method without any return value.

locate(): Promise<PatternControl>

Performs a match on the pattern object and returns the match result. If multiple results are matched, the one with the highest similarity (score) is returned.

• Return value: Promise<PatternControl> type, if no result is matched it will prompt 1001: ObjectNotExist.

modelProperties(): {[x: string]: any};

Returns the model properties of the pattern object, i.e. the cached properties generated when the pattern object is added, such as the path to the screenshot file, the coordinates of the area of the screenshot, etc.

• Return value: Promise<obeject>, returns an arbitrary object, for a normal pattern object, usually returns the result of an object shaped like the following:
  JavaScript

  {
      boundingRectangle: "[99,100,63,56]",
  	imagePath: "image.png"
  }

moveMouse(x?: number, y?: number, seconds?: number): Promise<void>

Move the mouse to the target position, you can specify the offset, the default position is the positive center of the pattern.

• x: (Optional) number type, horizontal coordinate, default is the horizontal center of the pattern object;
• y: (Optional) number type, vertical coordinate, default is the vertical center of the pattern object;
• seconds: (Optional) number type, the movement time, the smaller the value the faster the mouse moves;
• return value: is an asynchronous method without any return value.

score(): Promise<number>

Get the similarity result of the pattern object matching result, in the range [0,1).

• Return value: Promise<number>, a floating point number in the range 0~1, larger means the more similar the matching result is.

rect(): Promise<Rect>

Get the position and size information of the pattern object match result.

• Return value: Promise<Rect> object, contains the x, y, width, height information of the matched object.

wait(seconds: number): Promise<void>

If the result is matched within the specified time, the result will be returned and the next statement will continue to run; if the waiting time is exceeded and the result is not matched, an error will be thrown and an error message can be specified. The default is to wait 5 seconds.

• seconds: number type, wait time in seconds. The default value is 5 seconds.
• return value: Promise<void> type, an asynchronous method that does not return any result, or throws an error if the target pattern is not found after the timeout.

takeScreenshot(filePath?: string): Promise<string>

Match the target and take a screenshot, you can pass in the path to save the screenshot to the path location.

• filePath: (Optional) string type, the path to save the screenshot and the file name, e.g. ". /images/screenshot1.png".
• returnValue: Asynchronously returns the base64 string of the screenshot.

Code generation for Pattern objects

Pattern objects are usually first picked up through the model manager, and then the object methods can be generated directly by dragging and dropping them from the model manager into the code editor. For example, the following generation code.

await model.getPattern("Pattern1").click();

In addition to the above direct call, you can also pass in parameters to override the parameters of the same name in the pattern object:

await model.getPattern("PatternName", {resizeFactor: 2});

If the Pattern object needs to be reused in code, it can be assigned to a variable first and then called using the variable, For example:

let pattern = await model.getPattern("PatternName");
pattern.click();

Note that the positioning information in the Pattern object is cached, meaning that if the Pattern object is assigned to a variable as above, and after calling one of its operations (e.g. click), it will cache the image matching results in the object, at which point you will not redo the pattern matching by calling its other methods. This has the advantage of more efficient execution. However, if your control is not fixed on the screen, you need to call the locate method with this pattern variable to reposition it to ensure that you operate the control at the exact location.