Home > Mobile device testing > How to simulate touch actions using webdriver in Iphone or ipad

How to simulate touch actions using webdriver in Iphone or ipad

I have been working with webdriver for last 6-7 months now. The project that am working on now includes opening the website in iphone/ipad and testing the features in mobile safari browser. Webdriver and Grid 2.0 supports running ipad/iphone tests in Grid. This essentially means that you can remotely trigger the tests that needs to be run in the device. It looks really cool, when you see the tests being run remotely that too through the WIFI in these devices.

Everything was running smooth for some time, until my development decided to add touch actions to iphone. This meant that they added doubleTap, singleTap actions for the elements in the page. Its while trying to handle these events in Webdriver that I came to know that IPhoneDriver doesn’t support any touchevents. You cannot use the ui.interactions.actions classes too since its not supported in IphoneDriver. After trying the webdriver events out for some days without any success, I moved on to using javascript to trigger these events.

Javascripts were getting successfully executed in the mobile safari. All I need to get was the correct javascript. After some trial and error and help from lukeis in selenium (who maintains iphonedriver) I could figure out some of the touch actions. Given below is an utility class which has this implemented. This will work any locator as its using the position of the element in the page.

Note:- This will work “only” if the touchevents are implemented using  jQuery. Apps that were created using Sencha will not work using this code. For details see comments

public class CustomTouchAction {

//Javascripts to trigger the touch events. This will be appended to the javascript string

private static String DOUBLETAP= “.trigger(‘touchend’).trigger(‘touchend’);”;
private static String SINGLETAP=”.trigger(‘tap’);”;

private static String SWIPERIGHT=”.trigger(‘swiperight’);”;
private static String SWIPELEFT=”.trigger(‘swipeleft’);”;

private WebDriver driver;
public CustomTouchAction(WebDriver driver) {
this.driver = driver;
}

/**
* Return the x coordinate of the element in the screen
* @param elementToLocate
* @return
*/
private int getElementXPosition( String elementToLocate){
return driver.findElement(elementToLocate).getLocation().getX();
}

/**
* Return the y coordinate of the element in the screen
* @param elementToLocate
* @return
*/
private int getElementYPosition( String elementToLocate){
return driver.findElement(By.xpath(elementToLocate).getLocation().getY();
}

/**
* Get the common part of all touch action javascript trigger
* @param elementToTap
* @return
*/
private String addCommonScript(String elementToTap){
return “window.jQuery(document.elementFromPoint(”
+ getElementXPosition(elementToTap) + “,”
+ getElementYPosition(elementToTap) + “))”;
}

/**
* Simulate double tap on the element in the screen
* @param elementToTap
*/
public void doubleTap(String elementToTap){
String javascriptToExecute =addCommonScript(elementToTap) +DOUBLETAP;
executeScript(javascriptToExecute);
}

/**
* Simulate single tap on the element in the screen
* @param elementToTap
*/
public void singleTap(String elementToTap){
String javascriptToExecute = “window.jQuery(document.elementFromPoint(”
+ getElementXPosition(elementToTap) + “,”
+ getElementYPosition(elementToTap) + “))”+SINGLETAP;
executeScript(javascriptToExecute);
}

/**
* Simulate the swipe action. This will swipe from right to left from the position of the element
* @param elementToSwipe
*/
public void swipeLeft(String elementToSwipe){
String javascriptToExecute = “window.jQuery(document.elementFromPoint(”
+ getElementXPosition(elementToSwipe) + “,”
+ getElementYPosition(elementToSwipe) + “))”+SWIPELEFT;
executeScript(javascriptToExecute);
}

/**
* Simulate the swipe action. This will swipe from left to right from the position of the element
* @param elementToSwipe
*/
public void swipeRight(String elementToSwipe){
String javascriptToExecute = “window.jQuery(document.elementFromPoint(”
+ getElementXPosition(elementToSwipe) + “,”
+ getElementYPosition(elementToSwipe) + “))”+SWIPERIGHT;
executeScript(javascriptToExecute);
}

/**
* Execute the javascript code using javascript driver
* @param javascriptToExecute
*/
private void executeScript(String javascriptToExecute){

((JavascriptExecutor)driver).executeScript(javascriptToExecute);
}

}

Advertisements
  1. Suresh
    July 24, 2012 at 12:08 am

    Will this work for chromedriver?

    • AJ
      July 24, 2012 at 2:59 am

      I have not tested it, but I think this should work for all drivers since its using javascript.

  2. July 25, 2012 at 1:55 pm

    Hi, thanks for posting this!

    I’m trying to implement it but I keep getting a error message when I try to call one of the touch methods: org.openqa.selenium.WebDriverException: ‘undefined’ is not a function

    This seems to be thrown by the window.jQuery() part of the call – having stripped out the rest I’m still seeing the error. I’ve verified jQuery is available using a simple check: return (typeof jQuery == ‘undefined’);

    any ideas?

    • AJ
      July 25, 2012 at 2:29 pm

      Is the site that you are testing a public site which I can access?

    • July 26, 2012 at 12:43 pm

      running the below code – I don’t get any errors but the button does not get clicked/tapped. I made one change to your library – I take in a WebElement instead of the xpath descriptor.

      WebDriver driver = new IPhoneDriver();
      driver.get(“http://dev.sencha.com/deploy/touch/examples/production/kitchensink/#demo/nestedloading”);

      WebElement element = driver.findElement(By.id(“ext-button-1”));

      CustomTouchAction customTouchAction = new CustomTouchAction(driver);
      customTouchAction.singleTap(element);

      any ideas or guidance to how I can figure out how to simulate clicking on these buttons?

      • AJ
        July 30, 2012 at 4:04 pm

        I am trying to access the kitchensink site using ipad in useragent? when i try ‘ipad’ a blank page is displayed. Should I be giving something else?

      • Pavithra
        August 16, 2012 at 5:50 am

        Running the above; even I face the same issue. No error but its not clicked or tapped. Is there any thing else which I need to do ?
        Graham,
        Did you resolved this by any means?

  3. Pavithra
    August 14, 2012 at 6:01 pm

    Good blog!! I am trying to implement this. Could you please let me what package to be imported for singleTap method – Locator ?

    • AJ
      August 14, 2012 at 9:57 pm

      Pavithra, Its not locator but a String. Thank you for pointing out. Its updated now

  4. Pavithra
    August 16, 2012 at 12:01 pm

    Sorry I missed some ‘)’ in the function. I corrected it but now it is saying ‘undefined’ is not a function. Does this implementation expects jquery to be loaded by default? Our application doesn’t load jquery. So what should I do?

  5. August 16, 2012 at 12:28 pm

    I load it on the fly if it’s missing – this is what I ended up with:

  6. Pavithra
    August 16, 2012 at 1:00 pm

    Thanks a lot!! Will try it out and update.

    • August 16, 2012 at 1:01 pm

      I should mention – the above runs, but the clicks don’t actually do anything that I can see.

      • AJ
        August 16, 2012 at 2:27 pm

        Graham, will you able to to know how is the tap or double tap implemented in your application? For mine, the development team was using a doubleTap jQuery. So if you can get the implementation, we can look at what actions are they using to invoke singleTap. For iOS, its mostly touchEnd and touchStart

      • August 16, 2012 at 2:37 pm

        The app is using sencha touch 2 – so I think the details of this bug should apply:
        http://code.google.com/p/selenium/issues/detail?id=3833

      • AJ
        August 22, 2012 at 3:00 pm

        Was playing around a little with sencha applications and how it invokes the touch events. The reason why the above code doesn’t work is because sencha doesn’t use a jQuery touchevent. I think you would have to use Dispatcher.dispatchEvent() method to invoke the touch actions for sencha applications. I will add a note to the blog to convey that this will work only with applications that have jQuery touchevent implementation

  7. Pavithra
    August 21, 2012 at 11:49 am

    AJ/Graham, I tried the above by loading jquery on the fly. But as Graham mentioned it runs fine but the singletap or doubletap or even click event doesnt actually do anything. I confirmed with developer that it is using touchend event. Any idea?
    Graham, Did you got it work for your app?

    • AJ
      August 22, 2012 at 3:04 am

      Was playing around a little with sencha applications and how it invokes the touch events. The reason why the above code doesn’t work is because sencha doesn’t use a jQuery touchevent. I think you would have to use Dispatcher.dispatchEvent() method to invoke the touch actions for sencha applications. I will add a note to the blog to convey that this will work only with applications that have jQuery touchevent implementation

      • Pavithra
        August 22, 2012 at 5:59 am

        AJ, Thanks !! But as I mentioned before our app doesn’t use sencha touch. Our developer confirmed that it is using Hammer.js library and it uses tap event.

    • AJ
      August 22, 2012 at 2:59 pm

      Pavithra, you might have to change the javascript as below for hammer.js. I was able to simulate the event using JS in this demo site http://eightmedia.github.com/hammer.js/demo/ using the following JS
      window.jQuery(document.getElementById(‘container’)).trigger(‘tap’);

      Look for the events that your application is listening for. Hammer.js looks for the following events and all of them were working for me using the above javascript.

      Tap
      Double tap
      Hold
      Drag
      Swipe
      Transform (pinch)

  8. Pavithra
    August 23, 2012 at 5:13 am

    AJ, Thanks for the update!! So, does it mean that we should always find the element by javascript and we should not pass webelement to the jquery? Our app – dom structure doesn’t have unique ids to the elements, instead it has got class name. Below is the structure

    Today on Page

    Inbox

    Note: I have to access li data-fid=’Inbox’ element or element under it.

    I tried finding as webelement using xpath,//li[@data-fid=’Inbox’] and tried trigger(‘tap’) & .trigger(‘touchstart’).trigger(‘touchend’) & trigger(‘touchend’) events.

    Also, I tried finding using jquery:
    window.jQuery(document.getElementsByClassName(‘base-sprite img-inbox’)).trigger(‘tap’);
    window.jQuery(document.getElementsByClassName(‘base-sprite img-inbox’)).trigger(‘touchstart’).trigger(‘touchend’);

    Note: Our developer confirmed that it is listening to custom hammer event called ‘tap’ and they believes that touchstart followed by immediate touchend would cause hammer to understand as tap. [Which I anyway tried out]
    But none of the above worked.
    Any idea?

    • Pavithra
      August 23, 2012 at 5:16 am

      I posted the dom structure, which is not appearing above. could you please let me know your mail id so that i can mail you the same.

      • AJ
        August 23, 2012 at 2:53 pm

        Can you post the snippet in gist instead?

    • AJ
      August 23, 2012 at 2:56 pm

      How you find the element is upto you. You can use ID, ClassName or elementFromPoint (After finding the position of the element using webdriver). Are you able to find the element in the page using js?

  9. Pavithra
    September 11, 2012 at 6:35 am

    AJ,
    Sorry for late reply. Stuck with some other things. Yes after finding the webelement, I am able to find the element via javascript(using elementfrompoint) but only touch event is not getting triggered. I have a basic doubt – Does my ipad simulator supports touch event? My simulator is launched via xcode iWebdriver project and it is iOS simulator- iPAD / iOS 5.1.

    Because, I tried the demo site: http://eightmedia.github.com/hammer.js/demo/ and tried to perfrom single tap on div[@id=’container’] but again here, I dont see any tap being peformed. But you said – this site is working for you. So will it be problem with simulator? How do we make sure that simulator supports touch events?

    • Pavithra
      September 28, 2012 at 7:20 am

      AJ, I tried again on the demo site with .trigger(‘tap’) and it worked fine. So this site is not listening to touchstart and touchend. Its fine. But in my app, its listening to touchstart and touchend; however when i trigger(touchstart).trigger(‘touchend’) it doesnt have any effect. What could be the reason ?

  10. Aniket G
    December 12, 2012 at 7:20 am

    Hey , really nice post AJ. Could you please help me with dispatcher? Our app doesn’t use jquery. hence I tried the following but no luck

    var targetElement = document.getElementById(‘existing-customer-table-cell’);

    var evt = document.createEvent(‘TouchEvent’);
    evt.initTouchEvent(‘touchstart’, true, true);
    evt.view = window;
    evt.altKey = false;
    evt.ctrlKey = false;
    evt.shiftKey = false;
    evt.metaKey = false;

    var evt1 = document.createEvent(‘TouchEvent’);
    evt1.initTouchEvent(‘touchend’, true, true);
    evt1.view = window;
    evt1.altKey = false;
    evt1.ctrlKey = false;
    evt1.shiftKey = false;
    evt1.metaKey = false;
    targetElement.dispatchEvent(evt);
    targetElement.dispatchEvent(evt1);

  11. manoj n
    January 27, 2013 at 5:11 am

    Hey Aj
    Great Post! Actually I am new to selenium and working on something similar to yours. Can you please guide me as to how to run test scripts in ipad safari browser remotely? The iphone driver documentation was not very helpful, plus it launches your web app in UIWEBVIEW instead of safari. Any help would be appreciated 🙂

  12. Sharmila
    November 24, 2013 at 7:58 am

    Hi AJ,

    Thanks for the post. As Manoj , I am also new to selenium web driver. My tests are like do automating the ipad to open some sites/ streaming.

    Kindly share any documentation to start from scratch..

    Thanks in Advance,
    Sharmila

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: