Skip to content

高级控件交互方法

高级控件交互方法

简介

在 appium2.0 之前,在移动端设备上的触屏操作,单手指触屏和多手指触屏分别是由 TouchAction 类,Multiaction 类实现的。在 appium 2.0 之后,这 2 个方法将会被舍弃,需要改用 W3C actions。

在 W3C actions 当中,将输入源分为了三类:

w3c官网介绍

  • 键盘类 - Key
    • KeyDown:按下某个键
    • KeyUp:释放某个键
  • 指针类 - Pointer:Mouse 鼠标、Touch 触屏、Pen 笔触
    • PointerDown:按下鼠标键,或者触屏或者触屏笔触屏
    • PointerUp:松开鼠标键,或者手离开屏幕,或者触屏笔离开屏幕
    • PointerMove:移动到屏幕某个点
    • PointerCancel:删除某个指针操作
  • 空输入源 - None
    • pause:不做任何操作一段时间,或者动作的持续时间

使用场景

  • 点击和双击操作:模拟单击和双击操作。
  • 长按和释放:模拟长时间按住和释放操作。
  • 拖放操作:模拟拖放操作,将一个元素拖到另一个元素上。
  • 滑动操作:模拟滑动屏幕操作,如左右滑动、上下滑动。
  • 多点触控:模拟多点触控操作,如捏合放大、捏合缩小。

使用方法

Python 示例:

  1. 定义输入源
# 定义ActionChains实例
actions = ActionChains(driver)
# 定义输入源
actions.w3c_actions = ActionBuilder(
    driver, 
    mouse=PointerInput(interaction.POINTER_TOUCH, "touch")
)
  1. 按下:actions.w3c_actions.pointer_action.pointer_down()
  2. 拖拽:actions.w3c_actions.pointer_action.move_to_location(x, y)
  3. 抬起:actions.w3c_actions.pointer_action.release()
  4. 停顿:actions.w3c_actions.pointer_action.pause(0.5)
  5. 执行操作:actions.perform()

Java 示例:

  1. 定义输入源
// 定义输入源
PointerInput finger1 = new PointerInput(PointerInput.Kind.TOUCH, "finger1");
PointerInput finger2 = new PointerInput(PointerInput.Kind.TOUCH, "finger2");
// 创建指针序列
Sequence sequence1 = new Sequence(finger1, 1);
Sequence sequence2 = new Sequence(finger2, 2);
  1. 按下:sequence.addAction(finger.createPointerDown(PointerInput.MouseButtsequence.addAction(finger.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), x, y));
  2. 抬起:sequence.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));
  3. 停顿:actions.w3c_actions.pointer_action.pause(0.5)
  4. 执行操作:sequence.addAction(new Pause(finger, Duration.ofMillis(500)));

模拟两指缩放画面示例

  • 通过输入两个设备:touch
  • 分别执行反方向的长按,移动操作
  • 或者执行向中间靠拢的移动
  • 便能实现页面的缩放

Python 示例:

def Two_finger_move(driver):
    #创建actionchains对象
    actions = ActionChains(driver)
    actions.w3c_actions.devices=[]
    #创建输入设备
    finger1 = actions.w3c_actions.add_pointer_input('touch','finger1')
    finger2 = actions.w3c_actions.add_pointer_input('touch','finger2')
    #获取当前页面的长度和宽度
    width = driver.get_window_size()['width']
    height = driver.get_window_size()['height']
    #输入设备移动
    finger1.create_pointer_move(x=width*0.5,y=height*0.5)
    #输入设备移动
    finger2.create_pointer_move(x=width*0.5,y=height*0.5)
    #按下输入设备的鼠标左键
    finger1.create_pointer_down(MouseButton.LEFT)
    #按下输入设备的鼠标左键
    finger2.create_pointer_down(MouseButton.LEFT)
    #移动输入设备
    finger1.create_pointer_move(x=width*0.5,y=height*0.9)
    #移动输入设备
    finger2.create_pointer_move(x=width*0.5,y=height*0.1)
    #松开输入设备
    finger1.create_pointer_up(MouseButton.LEFT)
    #松开输入设备
    finger2.create_pointer_up(MouseButton.LEFT)
    #执行actions对象的动作序列
    actions.perform()

Java 示例:

@Test
public void twoFingerMove() {
    int width = driver.manage().window().getSize().getWidth();
    int height = driver.manage().window().getSize().getHeight();

    // 将高度转换为浮点数,并计算位置
    double midY = height * 0.5;
    double endY1 = height * 0.9;
    double endY2 = height * 0.1;

    // 创建 PointerInput 对象
    PointerInput finger1 = new PointerInput(PointerInput.Kind.TOUCH, "finger1");
    PointerInput finger2 = new PointerInput(PointerInput.Kind.TOUCH, "finger2");

    // 创建 Sequence 对象
    Sequence sequence1 = new Sequence(finger1, 1);
    sequence1.addAction(finger1.createPointerMove(Duration.ofMillis(0), PointerInput.Origin.viewport(), width / 2, (int) endY1));
    sequence1.addAction(finger1.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
    sequence1.addAction(finger1.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), width / 2, (int) midY));
    ;
    sequence1.addAction(finger1.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

    Sequence sequence2 = new Sequence(finger2, 2);
    sequence2.addAction(finger2.createPointerMove(Duration.ofMillis(0), PointerInput.Origin.viewport(), width / 2, (int) endY2));
    sequence2.addAction(finger2.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
    sequence2.addAction(finger2.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), width / 2, (int) midY));
    sequence2.addAction(finger2.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

    // 将两个 Sequence 添加到一个列表中
    List<Sequence> sequences = new ArrayList<>();
    sequences.add(sequence1);
    sequences.add(sequence2);

    // 执行两个 Sequence 同时执行
    driver.perform(sequences);

}

滑动解锁示例

  • 安装手势密码锁 app(TouchAction.apk),apk 网盘地址
  • 打开应用
  • 点击【设置手势】
  • 完成手势操作(如图)

实现手势滑动时,通常需要结合坐标,并可通过设置设备的输入选项,从界面中找到具体的坐标点。

手势滑动路径如下图所示:

Python 版本

class TestActionChains:

    def setup_class(self):
        # 设置启动参数
        caps = {
            "platformName": "Android",
            "appium:appPackage": "cn.kmob.screenfingermovelock",
            "appium:appActivity": "com.samsung.ui.FlashActivity",
            "appium:noReset": True,
            "appium:shouldTerminateApp": True,
        }
        # 初始化 driver
        self.driver = webdriver.Remote(
            'http://localhost:4723', 
            options=UiAutomator2Options().load_capabilities(caps)
        )
        # 设置隐式等待
        self.driver.implicitly_wait(15)

    def teartdown_class(self):
        # 退出应用程序
        self.driver.quit()

    def test_slide_to_unlock(self):
        # 点击设置手势
        self.driver.find_element(
            by=AppiumBy.ID, 
            value="cn.kmob.screenfingermovelock:id/patternTxt"
        ).click()
        print(self.driver.get_window_size())
        # 定义ActionChains实例
        actions = ActionChains(self.driver)
        # 定义输入源
        actions.w3c_actions = ActionBuilder(
            self.driver, 
            mouse=PointerInput(interaction.POINTER_TOUCH, "touch")
        )
        # 定义动作 pointer_down按下 pause暂停 release释放
        # 需要实现3个点之间的滑动,A->B 水平滑动 B—>C 竖直滑动
        bounds = self.driver.find_element(
            AppiumBy.ID, 
            'cn.kmob.screenfingermovelock:id/patternView'
        ).get_attribute('bounds')
        actions.w3c_actions.pointer_action.move_to_location(204, 377)
        actions.w3c_actions.pointer_action.pointer_down()
        actions.w3c_actions.pointer_action.move_to_location(930, 373)
        # 停顿0.5s 模拟在两个点之间进行拖拽操作
        actions.w3c_actions.pointer_action.pause(0.5)
        actions.w3c_actions.pointer_action.move_to_location(846, 1150)
        actions.w3c_actions.pointer_action.pause(0.5)
        actions.w3c_actions.pointer_action.release()
        # 执行操作
        actions.perform()
                # 获取【继续】按钮的 clickable 属性值
        result = self.driver.find_element(
            AppiumBy.ID, 
            "cn.kmob.screenfingermovelock:id/btnTwo"
        ).get_attribute("clickable")
        # 断言【继续按钮】可点击
        assert result == "true"

Java 示例:

import io.appium.java_client.AppiumBy;
import io.appium.java_client.android.AndroidDriver;
import io.appium.java_client.remote.MobileCapabilityType;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.interactions.PointerInput;
import org.openqa.selenium.interactions.Sequence;
import org.openqa.selenium.remote.DesiredCapabilities;

import java.net.MalformedURLException;
import java.net.URL;
import java.time.Duration;
import java.util.Collections;


public class TestActionChains {


    private static AndroidDriver driver;

    @BeforeAll
    public static void setUp() throws MalformedURLException {
        // 初始化capability
        DesiredCapabilities caps = new DesiredCapabilities();
        // 设置 app 安装的平台(Android、iOS)
        caps.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android");
        // 设置 app 的包名
        caps.setCapability("appPackage", "cn.kmob.screenfingermovelock");
        // 设置 app 的启动页
        caps.setCapability("appActivity", "com.samsung.ui.FlashActivity");
        // 设置 app 不清空缓存
        caps.setCapability("appium:noReset", true);
        // 设置 app 不重启
        caps.setCapability("appium:shouldTerminateApp", true);

        // 设置启动url
        URL remoteUrl = new URL("http://127.0.0.1:4723");
        // 初始化driver
        driver = new AndroidDriver(remoteUrl, caps);
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(15));
    }

    @AfterAll
    public static void teardownClass() {
        if (driver != null) {
            driver.quit();
        }
    }

    @Test
    public void testSlideToUnlock() {
        WebElement buttonEle = driver.findElement(AppiumBy.id("cn.kmob.screenfingermovelock:id/patternTxt"));
        buttonEle.click();

        // 创建 PointerInput
        PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger");

        // 创建 Sequence
        // 分辨率不同,坐标需要调整,demo 分辨率为 900 * 1600
        Sequence sequence = new Sequence(finger, 1);
        sequence.addAction(finger.createPointerMove(Duration.ofMillis(0), PointerInput.Origin.viewport(), 155, 220));
        sequence.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
        sequence.addAction(finger.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), 455, 220));
        sequence.addAction(finger.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), 755, 220));
        sequence.addAction(finger.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), 755, 520));
        sequence.addAction(finger.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), 755, 825));
        sequence.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

        // 执行 Sequence
        driver.perform(Collections.singletonList(sequence));

        // 获取【继续】按钮的 clickable 属性值
        WebElement continueButton = driver.findElement(AppiumBy.id("cn.kmob.screenfingermovelock:id/btnTwo"));
        String result = continueButton.getAttribute("clickable");

        // 断言【继续按钮】可点击
        assert result.equals("true");
    }
}

总结

  • ActionChains 用法
  • 滑动解锁示例