程序员最近都爱上了这个网站  程序员们快来瞅瞅吧!  it98k网:it98k.com

本站消息

站长简介/公众号

  出租广告位,需要合作请联系站长

+关注
已关注

分类  

暂无分类

标签  

暂无标签

日期归档  

2022-02(6)

2022-03(6)

【python】UI自动化-uiautomation

发布于2023-01-21 10:01     阅读(403)     评论(0)     点赞(12)     收藏(2)


0- 前言

做应用程序自动点击的记录,暂不完整,仅供参考。参考多位大佬文章,链接在后面

涉及工具

  1. inspect【应用程序元素定位辅助工具; UI SPY 这个软件好像也可以,见别人用过】
  2. uiautomation【本文自动化的重点】
  3. pyautogui【技术不到位,有些元素没法精确定位,用uiautomation获取旁边元素的位置坐标,再用这个模拟键鼠操作】

1- inspect

在这里插入图片描述

# --常用解析--

# Name,有些程序有,有些没有
Name:	"学习助手"

# ControlType,控件类型,查找窗口的时候 不同控件类型,用不同方法
ControlType:	UIA_WindowControlTypeId (0xC370)

# BoundingRectangle,边界矩形,显示这个窗口或按钮、标签、文本框  的 位置信息【左上x坐标,左上y坐标,右下x坐标,右下y坐标】
BoundingRectangle:	{l:88 t:110 r:358 b:309}

# 这个 查找窗口的时候 用 
ClassName:	"TkTopLevel"
    

How found:	Selected from tree...
Name:	"微信"
ControlType:	UIA_WindowControlTypeId (0xC370)
LocalizedControlType:	"窗口"
BoundingRectangle:	{l:-498 t:193 r:642 b:998}
IsEnabled:	true
IsOffscreen:	false
IsKeyboardFocusable:	false
HasKeyboardFocus:	false
AccessKey:	""
ProcessId:	2708
RuntimeId:	[2A.2406AE]
FrameworkId:	"Win32"
ClassName:	"WeChatMainWndForPC"
NativeWindowHandle:	0x2406AE
IsControlElement:	true
ProviderDescription:	"[pid:2400,providerId:0x2406AE Main:Nested [pid:2708,providerId:0x2406AE Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (IAccessible2) (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
AriaProperties:	""
IsPassword:	false
IsRequiredForForm:	false
IsDataValidForForm:	true
HelpText:	""
Culture:	0
IsDialog:	false
LegacyIAccessible.ChildId:	0
LegacyIAccessible.DefaultAction:	""
LegacyIAccessible.Description:	""
LegacyIAccessible.Help:	""
LegacyIAccessible.KeyboardShortcut:	""
LegacyIAccessible.Name:	"微信"
LegacyIAccessible.Role:	客户端 (0xA)
LegacyIAccessible.State:	正常 (0x0)
LegacyIAccessible.Value:	""
Transform.CanMove:	true
Transform.CanResize:	true
Transform.CanRotate:	false
Window.CanMaximize:	true
Window.CanMinimize:	true
Window.IsModal:	false
Window.IsTopmost:	false
Window.WindowInteractionState:	ReadyForUserInteraction (2)
Window.WindowVisualState:	Normal (0)
IsAnnotationPatternAvailable:	false
IsDragPatternAvailable:	false
IsDockPatternAvailable:	false
IsDropTargetPatternAvailable:	false
IsExpandCollapsePatternAvailable:	false
IsGridItemPatternAvailable:	false
IsGridPatternAvailable:	false
IsInvokePatternAvailable:	false
IsItemContainerPatternAvailable:	false
IsLegacyIAccessiblePatternAvailable:	true
IsMultipleViewPatternAvailable:	false
IsObjectModelPatternAvailable:	false
IsRangeValuePatternAvailable:	false
IsScrollItemPatternAvailable:	true
IsScrollPatternAvailable:	false
IsSelectionItemPatternAvailable:	false
IsSelectionPatternAvailable:	false
IsSpreadsheetItemPatternAvailable:	false
IsSpreadsheetPatternAvailable:	false
IsStylesPatternAvailable:	false
IsSynchronizedInputPatternAvailable:	false
IsTableItemPatternAvailable:	false
IsTablePatternAvailable:	false
IsTextChildPatternAvailable:	false
IsTextEditPatternAvailable:	false
IsTextPatternAvailable:	false
IsTextPattern2Available:	false
IsTogglePatternAvailable:	false
IsTransformPatternAvailable:	true
IsTransform2PatternAvailable:	false
IsValuePatternAvailable:	false
IsVirtualizedItemPatternAvailable:	false
IsWindowPatternAvailable:	true
IsCustomNavigationPatternAvailable:	false
IsSelectionPattern2Available:	false
FirstChild:	"" 窗格
LastChild:	"" 窗格
Next:	"【python】UI自动化-uiautomation.md• - Typora" 窗口
Previous:	"Home - 360极速浏览器X 21.0" 窗口
Other Props:	Object has no additional properties
Children:	"" 窗格
	"" 窗格
Ancestors:	"桌面 1" 窗格
	[ No Parent ]

2- uiautomation

一、uiautomation方法

方法备注
WindowContrl(searchDepth,ClassName,SubName)查找窗口中的程序,如果有中文则需用Unicode
window.Exists(maxSearchSeconds)来判断此窗口是否存在
EditControl(searchFromControl)查找编辑位置,找到后可用DoubleClick()来改变电脑的focus;edit.SetValue(“string”)输入值
Win32API.SendKeys(“string”)如果已在编辑位置,则可用此方法来输入值,{Ctrl}为ctrl键,其他类似;{@8}格式可输入8个@,对于数字也可实现此功能,但对于字母不能…;
MenuItemControl(searchFromControl,Name)查找菜单按钮
ComboBoxControl(searchFromControl,AutomationI)查找下拉框,然后在此基础上用Select(“name”)方法来选择需要的选项
import uiautomation
import subprocess

# 查找窗口中的程序,如果有中文则需用Unicode;可用
uiautomation.WindowContrl(searchDepth,ClassName,SubName) 

# 来判断此窗口是否存在
uiautomation.window.Exists(maxSearchSeconds)

# 查找编辑位置,找到后可用DoubleClick()来改变电脑的focus;edit.SetValue(“string”)输入值
uiautomation.EditControl(searchFromControl) 

# 查找菜单按钮
uiautomation.MenuItemControl(searchFromControl,Name) 

#  查找下拉框,然后在此基础上用Select(“name”)方法来选择需要的选项
uiautomation.ComboBoxControl(searchFromControl,AutomationI)

# 查找按钮
uiautomation.BottonControl(searchFromControl,Name,SubName) 
# Uiautomation元素获取方法如下

程序窗口:uiautomation.WindowControl
按钮:uiautomation.ButtonControl
文本:uiautomation.TextControl
输入窗口:uiautomation.EditControl
文档控件:uiautomation.DocumentControl
单选控件:uiautomation.CheckBoxControl
复选控件:uiautomation.ComboBoxControl
日历控件:uiautomation.CalendarControl

'''
ControlType为”ControlType.Window”
那么捕获窗口就用:uiautomation.WindowControl

如果ControlType为”ControlType.Text”
那么捕获窗口就用:uiautomation.TextControl

依次类推...
如果ControlType为”ControlType.Xxx”
那么捕获窗口就用:uiautomation.XxxControl

如果类型不明确,则使用uiautomation.Control
'''
# 常用的操作有
DoubleClick()双击
Click()单击
RightClick()右键点击
SendKeys()发送字符
SetValue()传值
# 三、对windows程序常用操作

subprocess.Popen(‘Name’)   用进程打开程序;

window.Close()         关闭窗口;

window.SetActive()     使用;

window.SetTopMost()     设置为顶层

window.ShowWindow(uiautomation.ShowWindow.Maximize) 窗口最大化

window.CaptureToImage(‘Notepad.png’) 截图;

3- 【实战】RustDesk软件自动 安装、指定服务器信息

下面代码引入的文件 的内容

192.168.1.78
vEJU4Fasd/asdadagsdaasdgadfasdasfasfaasffffffddddddddddddddda6W8=
rustdesk-1.1.9-putes.exe

---读取顺序---
---第1行 服务器ip---
---第2行 KEY---
---第3行 rustdesk安装包名字[要带拓展名]---

功能:自动安装rustdesk客户端,安装后开启"允许IP直接访问",执行期间不要使用键盘鼠标

# -*- coding: utf-8 -*-
# @Time : 2022/11/18 9:06
# @File : UI自动化.py
# @Software: PyCharm

import time
import uiautomation as auto
import subprocess

import pyautogui
import pyperclip
import os

f = open('key.txt', mode='r', encoding='utf-8')
key_list = f.read().split('\n')
f.close()

# 打开软件
subprocess.Popen(key_list[2])
auto.uiautomation.SetGlobalSearchTimeout(5)  # 设置全局搜索超时 5

# 获取
print('----开始安装--请勿操作键盘鼠标----')
win_0 = auto.WindowControl(searchDepth=1, Name="", ClassName='H-SMILE-FRAME')
win_0.ButtonControl(Name='同意并安装').Click()
# time.sleep(int(key_list[2]))
time.sleep(10)
for i in range(1, 20):
    print(f'{i}')
    if os.access(r'C:\Users\Public\Desktop\RustDesk.lnk', os.X_OK):
        time.sleep(5)

        try:
            win_0 = auto.WindowControl(searchDepth=1, Name="", ClassName='H-SMILE-FRAME')
            win_0.ButtonControl(Name='关闭').Click()
            time.sleep(3)
        except:
            pass
        break
    else:
        time.sleep(1)

# -------执行-----
print('打开软件')
subprocess.Popen(r"C:\Program Files\RustDesk\RustDesk.exe")
# os.system(r'start C:\"Program Files"\RustDesk\RustDesk.exe')
auto.uiautomation.SetGlobalSearchTimeout(5)  # 设置全局搜索超时 5

# 获取
win = auto.WindowControl(searchDepth=1, Name="", ClassName='H-SMILE-FRAME')
print(win.BoundingRectangle)

# 点击最大化
# win.ButtonControl(Name="最大化").Click()

# 控制的应用窗口前置
win.SetTopmost(True)

# 获取 三个点 位置
button = win.TextControl(Name='你的桌面可以通过下面的ID和密码访问。')
# print(button.BoundingRectangle)

# ---把三个点的位置转换成列表,并点击---
a = str(button.BoundingRectangle)

a = list(map(int, a[1:a.find(')')].split(',')))  # 把列表中的字符串转成整型

print(a)

# x,y 为 单击坐标,button 为 鼠标左或右键点击('left' / 'right')
pyautogui.click(a[2], a[3] + 20, button='left')
time.sleep(1)
# ---把三个点的位置转换成列表,并点击---END---

# 设置
win.TextControl(Name='允许IP直接访问').Click()

pyautogui.click(a[2], a[3] + 20, button='left')
time.sleep(1)
win.TextControl(Name='ID/中继服务器').Click()
time.sleep(1)


# pyautogui.hotkey('ctrl', 'a')  # 按下 ctrl + c 组合键。此次测试不生效
# ---------
def key_cv(c, v):
    """ 模拟执行快捷键 """
    pyautogui.keyDown(c)
    pyautogui.keyDown(v)
    pyautogui.keyUp(v)
    pyautogui.keyUp(c)


key_cv('ctrl', 'a')

pyperclip.copy(key_list[0])  # 复制
pyperclip.paste()  # 粘贴

key_cv('ctrl', 'v')

# ---------

time.sleep(0.5)
pyautogui.press('Tab')
pyautogui.press('Tab')
pyautogui.press('Tab')

# ---------
key_cv('ctrl', 'a')

pyperclip.copy(key_list[1])  # 复制
pyperclip.paste()  # 粘贴

key_cv('ctrl', 'v')

time.sleep(1)
pyautogui.press('tab')
pyautogui.press('tab')
time.sleep(0.5)
pyautogui.press('enter')

time.sleep(1)

win.ButtonControl(Name='关闭').Click()
print('--------执行结束--------')
time.sleep(5)

4- 参考文章

# inspect
https://blog.csdn.net/knighthood2001/article/details/124297008

# uiautomation
https://blog.csdn.net/yangzhichao_csdn/article/details/124799781
https://blog.csdn.net/chenmozhe22/article/details/106926071
https://blog.csdn.net/qq_21142893/article/details/84874406
    
# PyautoGui
https://blog.csdn.net/weixin_38640052/article/details/112387653

原文链接:https://blog.csdn.net/weixin_53337011/article/details/128037155



所属网站分类: 技术文章 > 博客

作者:天下

链接:https://www.pythonheidong.com/blog/article/1885289/8b0128215ba3536516bc/

来源:python黑洞网

任何形式的转载都请注明出处,如有侵权 一经发现 必将追究其法律责任

12 0
收藏该文
已收藏

评论内容:(最多支持255个字符)