Cleaned up

Increase unregister delay timeout

README.md

@@ -3,6 +3,7 @@ Docker-Android
 
 [![Build Status](https://travis-ci.org/butomo1989/docker-android.svg?branch=master)](https://travis-ci.org/butomo1989/docker-android)
 [![codecov](https://codecov.io/gh/butomo1989/docker-android/branch/master/graph/badge.svg)](https://codecov.io/gh/butomo1989/docker-android)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/3f000ffb97db45a59161814e1434c429)](https://www.codacy.com/app/butomo1989/docker-appium?utm_source=github.com&utm_medium=referral&utm_content=butomo1989/docker-appium&utm_campaign=Badge_Grade)
 
 Docker-Android is a docker image built to be used for everything related to mobile website testing and Android project.
 
@@ -13,7 +14,7 @@ Samsung Device               |  Google Device
 Purpose
 -------
 
-1. Run UI tests for mobile websites with ([appium])
+1. Run UI tests for mobile websites with [appium]
 2. Build Android project and run unit tests with the latest build-tools
 3. Run UI tests for Android applications with different frameworks ([appium], [espresso], [robotium], etc.)
 4. Run monkey / stress tests
@@ -101,9 +102,6 @@ docker run --privileged -d -p 6080:6080 -p 5554:5554 -p 5555:5555 -p 4723:4723 -
 
 ### Connect to Selenium Grid
 
-![][selenium grid]
-![][devices are connected with selenium grid]
-
 It is also possible to connect appium server that run inside docker-android with selenium grid by passing following environment variables:
 
 - CONNECT\_TO\_GRID=True
@@ -112,8 +110,12 @@ It is also possible to connect appium server that run inside docker-android with
 - SELENIUM_HOST="\<host\_ip\_address>"
 - SELENIUM_PORT=\<port\_number>
 
+To run tests for mobile browser, following parameter can be passed:
+
+- MOBILE\_WEB\_TEST=True
+
 ```bash
-docker run --privileged -d -p 6080:6080 -p 4723:4723 -p 5554:5554 -p 5555:5555 -e DEVICE="Samsung Galaxy S6" -e APPIUM=True -e CONNECT_TO_GRID=True -e APPIUM_HOST="127.0.0.1" -e APPIUM_PORT=4723 -e SELENIUM_HOST="172.17.0.1" -e SELENIUM_PORT=4444 --name android-container butomo1989/docker-android-x86-7.1.1
+docker run --privileged -d -p 6080:6080 -p 4723:4723 -p 5554:5554 -p 5555:5555 -e DEVICE="Samsung Galaxy S6" -e APPIUM=True -e CONNECT_TO_GRID=True -e APPIUM_HOST="127.0.0.1" -e APPIUM_PORT=4723 -e SELENIUM_HOST="172.17.0.1" -e SELENIUM_PORT=4444 -e MOBILE_WEB_TEST=True --name android-container butomo1989/docker-android-x86-7.1.1
 ```
 
 ### Share Volume
@@ -124,6 +126,17 @@ If you want to use appium to test UI of your android application, you need to sh
 docker run --privileged -d -p 6080:6080 -p 4723:4723 -p 5554:5554 -p 5555:5555 -v $PWD/example/sample_apk:/root/tmp -e DEVICE="Nexus 5" -e APPIUM=True -e CONNECT_TO_GRID=True -e APPIUM_HOST="127.0.0.1" -e APPIUM_PORT=4723 -e SELENIUM_HOST="172.17.0.1" -e SELENIUM_PORT=4444 --name android-container butomo1989/docker-android-x86-7.1.1
 ```
 
+### Docker-Compose
+
+![][compose]
+![][connected_devices]
+
+There is [example of compose file] to run complete selenium grid and docker-android container as nodes. [docker-compose] version [1.13.0] or higher is required to be able to execute that compose file.
+
+```bash
+docker-compose up -d
+```
+
 Control android emulator outside container
 ------------------------------------------
 
@@ -186,7 +199,10 @@ docker exec -it android-container tail -f /var/log/supervisor/docker-android.std
 [robotium]: <https://github.com/RobotiumTech/robotium>
 [docker android samsung]: <images/docker_android_samsung.png>
 [docker android nexus]: <images/docker_android_nexus.png>
-[selenium grid]: <images/selenium_grid.png>
-[devices are connected with selenium grid]: <images/connected_with_grid.png>
+[compose]: <images/compose.png>
+[connected_devices]: <images/connected_devices.png>
+[example of compose file]: <docker-compose.yml>
+[docker-compose]: <https://docs.docker.com/compose/install/>
+[1.13.0]: <https://github.com/docker/compose/releases/tag/1.13.0>
 [adb_connection]: <images/adb_connection.png>
 [sms]: <images/SMS.png>

docker-compose.yml

@@ -0,0 +1,60 @@
+# Note: It requires docker-compose 1.13.0
+#
+# Usage: docker-compose up -d
+version: "2.2"
+
+services:
+  # Selenium hub
+  selenium_hub:
+    image: selenium/hub:3.4.0
+    ports:
+      - "4444:4444"
+
+  # Docker-Android for Android application testing
+  nexus_7.1.1:
+    image: butomo1989/docker-android-x86-7.1.1
+    privileged: true
+    # Change path of apk that you want to test. I use sample_apk that I provide in folder "example"
+    volumes:
+      - $PWD/example/sample_apk:/root/tmp
+    # Increase scale number if needed
+    scale: 1
+    ports:
+      - 6080
+    environment:
+      - DEVICE=Nexus 5
+      - CONNECT_TO_GRID=True
+      - APPIUM=true
+      - SELENIUM_HOST=selenium_hub
+
+  # Docker-Android for mobile website testing with chrome browser
+  # Chrome browser exists only for version 7.0 and 7.1.1
+  samsung_galaxy_web_7.1.1:
+    image: butomo1989/docker-android-x86-7.1.1
+    privileged: true
+    # Increase scale number if needed
+    scale: 1
+    ports:
+      - 6080
+    environment:
+      - DEVICE=Samsung Galaxy S6
+      - CONNECT_TO_GRID=True
+      - APPIUM=true
+      - SELENIUM_HOST=selenium_hub
+      - MOBILE_WEB_TEST=True
+
+  # Docker-Android for mobile website testing with default browser
+  # Default browser exists only for version 5.0.1, 5.1.1 and 6.0
+  samsung_galaxy_web_5.1.1:
+    image: butomo1989/docker-android-x86-5.1.1
+    privileged: true
+    # Increase scale number if needed
+    scale: 1
+    ports:
+      - 6080
+    environment:
+      - DEVICE=Samsung Galaxy S6
+      - CONNECT_TO_GRID=True
+      - APPIUM=true
+      - SELENIUM_HOST=selenium_hub
+      - MOBILE_WEB_TEST=True

docker/Emulator_arm

@@ -0,0 +1,171 @@
+FROM ubuntu:16.04
+
+ENV DEBIAN_FRONTEND noninteractive
+
+#=============
+# Set WORKDIR
+#=============
+WORKDIR /root
+
+#==================
+# General Packages
+#------------------
+# wget
+#   Network downloader
+# unzip
+#   Unzip zip file
+# curl
+#   Transfer data from or to a server
+# xterm
+#   Terminal emulator
+# supervisor
+#   Process manager
+# openjdk-8-jdk
+#   Java
+# libqt5webkit5
+#   Web content engine (Fix issue in Android)
+# socat
+#   Port forwarder
+# libgconf-2-4
+#   Required package for chrome and chromedriver to run on Linux
+#------------------
+#  NoVNC Packages
+#------------------
+# xvfb
+#   X virtual framebuffer
+# x11vnc
+#   VNC server for X display
+# openbox
+#   Windows manager
+# menu
+#   Debian menu
+# python-numpy
+#   Numpy, For faster performance: https://github.com/novnc/websockify/issues/77
+# net-tools
+#   Netstat
+#==================
+RUN apt-get -qqy update && apt-get -qqy install --no-install-recommends \
+    wget \
+    unzip \
+    curl \
+    xterm \
+    supervisor \
+    openjdk-8-jdk \
+    libqt5webkit5 \
+    socat \
+    libgconf-2-4 \
+    xvfb \
+    x11vnc \
+    openbox \
+    menu \
+    python-numpy \
+    net-tools \
+ && rm -rf /var/lib/apt/lists/*
+
+#=======
+# noVNC
+# Use same commit id that docker-selenium uses
+# https://github.com/elgalu/docker-selenium/blob/236b861177bd2917d864e52291114b1f5e4540d7/Dockerfile#L412-L413
+#=======
+ENV NOVNC_SHA="b403cb92fb8de82d04f305b4f14fa978003890d7" \
+    WEBSOCKIFY_SHA="558a6439f14b0d85a31145541745e25c255d576b"
+RUN  wget -nv -O noVNC.zip "https://github.com/kanaka/noVNC/archive/${NOVNC_SHA}.zip" \
+ && unzip -x noVNC.zip \
+ && rm noVNC.zip  \
+ && mv noVNC-${NOVNC_SHA} noVNC \
+ && wget -nv -O websockify.zip "https://github.com/kanaka/websockify/archive/${WEBSOCKIFY_SHA}.zip" \
+ && unzip -x websockify.zip \
+ && mv websockify-${WEBSOCKIFY_SHA} ./noVNC/utils/websockify \
+ && rm websockify.zip \
+ && ln noVNC/vnc_auto.html noVNC/index.html
+
+#=====================
+# Install Android SDK
+#=====================
+ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/jre
+ENV PATH ${PATH}:${JAVA_HOME}/bin
+
+ENV SDK_VERSION=25.2.3 \
+    ANDROID_HOME=/root
+RUN wget -nv -O android.zip https://dl.google.com/android/repository/tools_r${SDK_VERSION}-linux.zip \
+ && unzip android.zip && rm android.zip
+ENV PATH ${PATH}:${ANDROID_HOME}/tools
+RUN echo y | android update sdk --no-ui -a --filter platform-tools
+ENV PATH ${PATH}:${ANDROID_HOME}/platform-tools
+
+#====================================
+# Install latest nodejs, npm, appium
+#====================================
+RUN curl -sL https://deb.nodesource.com/setup_7.x | bash - \
+ && apt-get -qqy update && apt-get -qqy install nodejs && rm -rf /var/lib/apt/lists/*
+ENV APPIUM_VERSION 1.6.3
+RUN npm install -g appium@$APPIUM_VERSION && npm cache clean
+
+#======================
+# Install SDK packages
+#======================
+ARG ANDROID_VERSION=5.0.1
+ARG BUILD_TOOL=25.0.3
+ARG API_LEVEL=21
+ARG PROCESSOR=x86
+ARG SYS_IMG=x86_64
+ARG IMG_TYPE=google_apis
+ARG BROWSER=android
+ENV ANDROID_VERSION=$ANDROID_VERSION \
+    BUILD_TOOL=$BUILD_TOOL \
+    API_LEVEL=$API_LEVEL \
+    PROCESSOR=$PROCESSOR \
+    SYS_IMG=$SYS_IMG \
+    IMG_TYPE=$IMG_TYPE \
+    BROWSER=$BROWSER
+RUN echo y | android update sdk --no-ui -a --filter build-tools-${BUILD_TOOL}
+ENV PATH ${PATH}:${ANDROID_HOME}/build-tools
+
+RUN rm ${ANDROID_HOME}/tools/emulator \
+ && ln -s ${ANDROID_HOME}/tools/emulator64-${PROCESSOR} ${ANDROID_HOME}/tools/emulator
+RUN echo y | android update sdk --no-ui -a -t android-${API_LEVEL},sys-img-${SYS_IMG}-${IMG_TYPE}-${API_LEVEL}
+
+#================================================
+# noVNC Default Configurations
+# These Configurations can be changed through -e
+#================================================
+ENV DISPLAY=:0 \
+    SCREEN=0 \
+    SCREEN_WIDTH=1600 \
+    SCREEN_HEIGHT=900 \
+    SCREEN_DEPTH=16 \
+    LOCAL_PORT=5900 \
+    TARGET_PORT=6080 \
+    TIMEOUT=1 \
+    LOG_PATH=/var/log/supervisor
+
+#===============
+# Expose Ports
+#---------------
+# 4723
+#   Appium port
+# 6080
+#   noVNC port
+# 5554
+#   Emulator port
+# 5555
+#   ADB connection port
+#===============
+EXPOSE 4723 6080 5554 5555
+
+#======================
+# Add Emulator Devices
+#======================
+COPY devices /root/devices
+
+#===================
+# Run docker-appium
+#===================
+COPY src /root/src
+COPY supervisord.conf /root/
+RUN chmod -R +x /root/src && chmod +x /root/supervisord.conf
+
+HEALTHCHECK --interval=2s --timeout=600s --retries=1 \
+    CMD adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 1; done'
+
+CMD /usr/bin/supervisord --configuration supervisord.conf

docker/Emulator_x86

@@ -123,12 +123,14 @@ ARG API_LEVEL=21
 ARG PROCESSOR=x86
 ARG SYS_IMG=x86_64
 ARG IMG_TYPE=google_apis
+ARG BROWSER=android
 ENV ANDROID_VERSION=$ANDROID_VERSION \
     BUILD_TOOL=$BUILD_TOOL \
     API_LEVEL=$API_LEVEL \
     PROCESSOR=$PROCESSOR \
     SYS_IMG=$SYS_IMG \
-    IMG_TYPE=$IMG_TYPE
+    IMG_TYPE=$IMG_TYPE \
+    BROWSER=$BROWSER
 RUN echo y | android update sdk --no-ui -a --filter build-tools-${BUILD_TOOL}
 ENV PATH ${PATH}:${ANDROID_HOME}/build-tools
 
@@ -176,7 +178,7 @@ COPY src /root/src
 COPY supervisord.conf /root/
 RUN chmod -R +x /root/src && chmod +x /root/supervisord.conf
 
-HEALTHCHECK --interval=2s --timeout=600s --retries=1 \
+HEALTHCHECK --interval=2s --timeout=40s --retries=1 \
     CMD adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 1; done'
 
 CMD /usr/bin/supervisord --configuration supervisord.conf

docker/Real_device

@@ -0,0 +1 @@
+# TODO

example/android/python/app_simple.py

@@ -9,10 +9,12 @@ class SimpleAndroidUITests(unittest.TestCase):
         desired_caps = {
             'platformName': 'Android',
             'deviceName': 'Android Emulator',
+            'automationName': 'UIAutomator2',
             'app': '/root/tmp/sample_apk_debug.apk',
-            'avd': 'samsung_galaxy_s6_7.1.1'
+            'browserName': 'android',
+            'avd': 'nexus_5_7.1.1'
         }
-        self.driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)
+        self.driver = webdriver.Remote('http://127.0.0.1:4444/wd/hub', desired_caps)
 
     def tearDown(self):
         self.driver.quit()

example/android/python/msite_simple_chrome.py

@@ -13,9 +13,10 @@ class MSiteChromeAndroidUITests(unittest.TestCase):
             'deviceName': 'Android Emulator',
             'appPackage': 'com.android.chrome',
             'appActivity': 'com.google.android.apps.chrome.Main',
-            'avd': 'samsung_galaxy_s6_7.1.1'
+            'browserName': 'chrome',
+            'ignore-certificate-errors': True
         }
-        self.driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)
+        self.driver = webdriver.Remote('http://127.0.0.1:4444/wd/hub', desired_caps)
 
     def test_open_url(self):
         self.driver.get('http://targeturl.com')

example/android/python/msite_simple_default_browser.py

@@ -13,9 +13,9 @@ class MSiteDefaultBrowserAndroidUITests(unittest.TestCase):
             'deviceName': 'Android Emulator',
             'appPackage': 'com.android.browser',
             'appActivity': 'com.android.browser.BrowserActivity',
-            'avd': 'samsung_galaxy_s6_6.0'
+            'browserName': 'browser'
         }
-        self.driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)
+        self.driver = webdriver.Remote('http://127.0.0.1:4444/wd/hub', desired_caps)
 
     def test_open_url(self):
         self.driver.get('http://targeturl.com')

release.sh

@@ -103,12 +103,20 @@ function build() {
 
     # Build docker image(s)
     for p in "${processors[@]}"; do
+        if [ "$p" == "x86" ]; then
+            FILE_NAME=docker/Emulator_x86
+        else
+            FILE_NAME=docker/Emulator_arm
+        fi
+
         for v in "${versions[@]}"; do
-            # Find image type
-            if [ "$v" == "5.0.1" ] || [ "$v" == "5.1.1" ]; then
+            # Find image type and default web browser
+            if [ "$v" == "5.0.1" ] || [ "$v" == "5.1.1" ] || [ "$v" == "6.0" ]; then
                 IMG_TYPE=android
+                BROWSER=browser
             else
                 IMG_TYPE=google_apis
+                BROWSER=chrome
             fi
             echo "[BUILD] IMAGE TYPE: $IMG_TYPE"
             level=${list_of_levels[$v]}
@@ -118,12 +126,13 @@ function build() {
             image_version="$IMAGE-$p-$v:$RELEASE"
             image_latest="$IMAGE-$p-$v:latest"
             echo "[BUILD] Image name: $image_version and $image_latest"
+            echo "[BUILD] Dockerfile: $FILE_NAME"
             docker build -t $image_version --build-arg ANDROID_VERSION=$v --build-arg BUILD_TOOL=$LATEST_BUILD_TOOL \
             --build-arg API_LEVEL=$level --build-arg PROCESSOR=$p --build-arg SYS_IMG=$sys_img \
-            --build-arg IMG_TYPE=$IMG_TYPE .
+            --build-arg IMG_TYPE=$IMG_TYPE --build-arg BROWSER=$BROWSER -f $FILE_NAME .
             docker build -t $image_latest --build-arg ANDROID_VERSION=$v --build-arg BUILD_TOOL=$LATEST_BUILD_TOOL \
             --build-arg API_LEVEL=$level --build-arg PROCESSOR=$p --build-arg SYS_IMG=$sys_img \
-            --build-arg IMG_TYPE=$IMG_TYPE .
+            --build-arg IMG_TYPE=$IMG_TYPE --build-arg BROWSER=$BROWSER -f $FILE_NAME .
         done
     done
 }

src/app.py

@@ -26,7 +26,7 @@ def get_or_raise(env: str) -> str:
     return env_value
 
 
-def str_to_bool(str: str) -> bool:
+def convert_str_to_bool(str: str) -> bool:
     """
     Convert string to boolean.
 
@@ -100,24 +100,28 @@ def appium_run(avd_name: str):
     :param avd_name: Name of android virtual device / emulator
     """
     cmd = 'appium'
+    local_ip = os.popen('ifconfig eth0 | grep \'inet addr:\' | cut -d: -f2 | awk \'{ print $1}\'').read().strip()
 
-    grid_connect = str_to_bool(str(os.getenv('CONNECT_TO_GRID', False)))
+    grid_connect = convert_str_to_bool(str(os.getenv('CONNECT_TO_GRID', False)))
     logger.info('Connect to selenium grid? {connect}'.format(connect=grid_connect))
     if grid_connect:
         try:
-            appium_host = os.getenv('APPIUM_HOST', '127.0.0.1')
+            mobile_web_test = convert_str_to_bool(str(os.getenv('MOBILE_WEB_TEST', False)))
+            default_web_browser = os.getenv('BROWSER')
+            appium_host = os.getenv('APPIUM_HOST', local_ip)
             appium_port = int(os.getenv('APPIUM_PORT', 4723))
             selenium_host = os.getenv('SELENIUM_HOST', '172.17.0.1')
             selenium_port = int(os.getenv('SELENIUM_PORT', 4444))
-            create_node_config(avd_name, appium_host, appium_port, selenium_host, selenium_port)
+            browser_name = default_web_browser if mobile_web_test else 'android'
+            create_node_config(avd_name, browser_name, appium_host, appium_port, selenium_host, selenium_port)
             cmd += ' --nodeconfig {file}'.format(file=CONFIG_FILE)
         except ValueError as v_err:
             logger.error(v_err)
-    titel = 'Appium Server'
-    subprocess.check_call('xterm -T "{titel}" -n "{titel}" -e \"{cmd}\"'.format(titel=titel, cmd=cmd), shell=True)
+    title = 'Appium Server'
+    subprocess.check_call('xterm -T "{title}" -n "{title}" -e \"{cmd}\"'.format(title=title, cmd=cmd), shell=True)
 
 
-def create_node_config(avd_name: str, appium_host: str, appium_port: int, selenium_host: str, selenium_port: int):
+def create_node_config(avd_name: str, browser_name: str, appium_host: str, appium_port: int, selenium_host: str, selenium_port: int):
     """
     Create custom node config file in json format to be able to connect with selenium server.
 
@@ -133,7 +137,8 @@ def create_node_config(avd_name: str, appium_host: str, appium_port: int, seleni
                 'platform': 'Android',
                 'platformName': 'Android',
                 'version': ANDROID_VERSION,
-                'browserName': avd_name,
+                'browserName': browser_name,
+                'deviceName': avd_name,
                 'maxInstances': 1,
             }
         ],
@@ -148,7 +153,8 @@ def create_node_config(avd_name: str, appium_host: str, appium_port: int, seleni
             'register': True,
             'registerCycle': 5000,
             'hubHost': selenium_host,
-            'hubPort': selenium_port
+            'hubPort': selenium_port,
+            'unregisterIfStillDownAfter': 120000
         }
     }
     logger.info('Appium node config: {config}'.format(config=config))
@@ -170,7 +176,7 @@ def run():
     cmd = 'emulator -avd {name}'.format(name=avd_name)
     subprocess.Popen(cmd.split())
 
-    appium = str_to_bool(str(os.getenv('APPIUM', False)))
+    appium = convert_str_to_bool(str(os.getenv('APPIUM', False)))
     if appium:
         logger.info('Run appium server...')
         appium_run(avd_name)

src/tests/__init__.py

@@ -21,17 +21,17 @@ class TestApp(TestCase):
             app.get_or_raise('ENV_2')
 
     def test_valid_bool(self):
-        self.assertEqual(app.str_to_bool('True'), True)
-        self.assertEqual(app.str_to_bool('t'), True)
-        self.assertEqual(app.str_to_bool('1'), True)
-        self.assertEqual(app.str_to_bool('YES'), True)
+        self.assertEqual(app.convert_str_to_bool('True'), True)
+        self.assertEqual(app.convert_str_to_bool('t'), True)
+        self.assertEqual(app.convert_str_to_bool('1'), True)
+        self.assertEqual(app.convert_str_to_bool('YES'), True)
 
     def test_invalid_bool(self):
-        self.assertEqual(app.str_to_bool(''), False)
-        self.assertEqual(app.str_to_bool('test'), False)
+        self.assertEqual(app.convert_str_to_bool(''), False)
+        self.assertEqual(app.convert_str_to_bool('test'), False)
 
     def test_invalid_format(self):
-        self.assertEqual(app.str_to_bool(True), None)
+        self.assertEqual(app.convert_str_to_bool(True), None)
 
     @mock.patch('src.app.prepare_avd')
     @mock.patch('subprocess.Popen')

src/tests/test_appium.py

@@ -44,7 +44,7 @@ class TestAppium(TestCase):
     def test_config_creation(self):
         from src import CONFIG_FILE
         self.assertFalse(os.path.exists(CONFIG_FILE))
-        app.create_node_config('test', '127.0.0.1', 4723, '127.0.0.1', 4444)
+        app.create_node_config('test', 'android', '127.0.0.1', 4723, '127.0.0.1', 4444)
         self.assertTrue(os.path.exists(CONFIG_FILE))
         os.remove(CONFIG_FILE)