Merge pull request #104 from devilankur18/master

Added device_id support via ENV variable or auto uuid values

README.md

@@ -12,7 +12,7 @@ Docker-Android is a docker image built to be used for everything related to mobi
 
 <p align="center">
    <a href="https://youtu.be/pQdpjuYwvp8"><img src="./images/appiumconf2018.png" alt="Appium Conference 2018" width="600"></a>
-</p> 
+</p>
 
 Emulator - Samsung Device    | Emulator - Nexus Device     | Real Device  
 :---------------------------:|:---------------------------:|:---------------------------:
@@ -41,15 +41,17 @@ Advantages compare with other docker-android projects
 List of Docker images
 ---------------------
 
-|Supported OS   |Android version   |API level   |Image name   |Image status   |
-|:---|:---|:---|:---|:---:|
-|Linux|5.0.1|21|butomo1989/docker-android-x86-5.0.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.0.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.0.1 "Get your own image badge on microbadger.com")|
-|Linux|5.1.1|22|butomo1989/docker-android-x86-5.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.1.1 "Get your own image badge on microbadger.com")|
-|Linux|6.0|23|butomo1989/docker-android-x86-6.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-6.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-6.0 "Get your own image badge on microbadger.com")|
-|Linux|7.0|24|butomo1989/docker-android-x86-7.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.0 "Get your own image badge on microbadger.com")|
-|Linux|7.1.1|25|butomo1989/docker-android-x86-7.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.1.1 "Get your own image badge on microbadger.com")|
-|All |-|-|butomo1989/docker-android-real-device|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-real-device.svg)](https://microbadger.com/images/butomo1989/docker-android-real-device "Get your own image badge on microbadger.com")|
-|Linux|All|All|butomo1989/docker-android-genymotion|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-genymotion.svg)](https://microbadger.com/images/butomo1989/docker-android-genymotion "Get your own image badge on microbadger.com")|
+|OS   |Android version   |API   |Browser   |Browser version   |Chromedriver version   |Image   |Size   |
+|:---|:---|:---|:---|:---|:---|:---|:---|
+|Linux|5.0.1|21|browser|37.0|2.12|butomo1989/docker-android-x86-5.0.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.0.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.0.1 "Get your own image badge on microbadger.com")|
+|Linux|5.1.1|22|browser|39.0|2.13|butomo1989/docker-android-x86-5.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.1.1 "Get your own image badge on microbadger.com")|
+|Linux|6.0|23|browser|44.0|2.18|butomo1989/docker-android-x86-6.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-6.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-6.0 "Get your own image badge on microbadger.com")|
+|Linux|7.0|24|chrome|51.0|2.23|butomo1989/docker-android-x86-7.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.0 "Get your own image badge on microbadger.com")|
+|Linux|7.1.1|25|chrome|55.0|2.28|butomo1989/docker-android-x86-7.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.1.1 "Get your own image badge on microbadger.com")|
+|Linux|8.0|26|chrome|58.0|2.31|butomo1989/docker-android-x86-8.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-8.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-8.0 "Get your own image badge on microbadger.com")|
+|Linux|8.1|27|chrome|61.0|2.33|butomo1989/docker-android-x86-8.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-8.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-8.1 "Get your own image badge on microbadger.com")|
+|All |-|-|-|-|-|butomo1989/docker-android-real-device|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-real-device.svg)](https://microbadger.com/images/butomo1989/docker-android-real-device "Get your own image badge on microbadger.com")|
+|Linux|All|All|All|All|All|butomo1989/docker-android-genymotion|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-genymotion.svg)](https://microbadger.com/images/butomo1989/docker-android-genymotion "Get your own image badge on microbadger.com")|
 
 List of Devices
 ---------------
@@ -79,7 +81,7 @@ Quick Start
 		docker run --privileged -d -p 6080:6080 -p 5554:5554 -p 5555:5555 -e DEVICE="Samsung Galaxy S6" --name android-container butomo1989/docker-android-x86-8.1
 		```
 
-	- For ***OSX*** and ***Windows OS***, please use Virtual Machine that support Virtualization
+	- For ***OSX*** and ***Windows OS***, please use Virtual Machine that support Virtualization with Ubuntu OS
 
 
 2. Verify the ip address of docker host.
@@ -142,6 +144,24 @@ There is [example of compose file] to run complete selenium grid and docker-andr
 ```bash
 docker-compose up -d
 ```
+### Google Play Services
+
+The Google play services (v12.8.74) will be downloaded from [apklinker](https://www.apklinker.com/wp-content/uploads/uploaded_apk/5b51570a214a8/com.google.android.gms_12.8.74-040700-204998136_12874026_MinAPI23_(x86)(nodpi)_apklinker.com.apk)  in [utils bash file] file when emulator booted . you can edit [utils bash file] as you need any adb stuffs as the following to disable emulators animation to increase performance:
+
+      adb shell "settings put global window_animation_scale 0.0"
+      adb shell "settings put global transition_animation_scale 0.0"
+      adb shell "settings put global animator_duration_scale 0.0"
+
+ mount it as follow in docker-compose file :
+
+     volumes:
+      - $PWD/videos:/tmp/video
+      - $PWD/src/utils.sh:/root/src/utils.sh
+
+or when run directly
+```bash
+docker run --privileged -d -p 6080:6080 -p 4723:4723 -p 5554:5554 -p 5555:5555 -v $PWD/src/utils.sh:/root/src/utils.sh -v $PWD/example/sample_apk:/root/tmp -e DEVICE="Nexus 5" -e APPIUM=true -e CONNECT_TO_GRID=true -e APPIUM_HOST="127.0.0.1" -e APPIUM_PORT=4723 -e SELENIUM_HOST="172.17.0.1" -e SELENIUM_PORT=4444 --name android-container butomo1989/docker-android-x86-8.1      
+```
 
 Build Android project
 ---------------------
@@ -179,9 +199,9 @@ Genymotion
 
 ![Genymotion](images/logo_genymotion.png)
 
-Docker-Android supports [Genymotion Cloud]. 
+Docker-Android supports [Genymotion Cloud].
 
-You can easily scale your Appium tests on Genymotion Android virtual devices in the cloud. 
+You can easily scale your Appium tests on Genymotion Android virtual devices in the cloud.
 Use [device.json] to define the device to start. You can specify the port on which the device will start so you don't need to change the device name in your tests every time you need to run those tests. Then run following command
 
 ```bash
@@ -210,6 +230,15 @@ adb connect <docker-machine-ip-address>:5555
 
 **Note:** You need to have Android Debug Bridge (adb) installed in your host machine.
 
+Get rid of unauthorized device popup while connecting to the emulator outside the container
+------------------------------------------
+
+In order to connect to the emulator using already generated adbkey on your machine you will need to mount your `.android` folder to the container (this is needed only when you work with emulator based on playstore image):
+
+```
+docker run -v <local Android key directory>:/root/.android/ ...
+```
+
 SMS Simulation
 --------------
 
@@ -268,7 +297,7 @@ The following instructions are used for OS X. You'll need [docker-machine-parall
     ```bash
     $ docker-machine create --driver=parallels prl-dev
     ```
-    
+
     This utility `docker-machine-parallels` will fetch boot2docker.iso to create a vm of VMWare fusion or Parallels Desktop. When the vm is created, you'll see it's booted with VMWare fusion or Parallels Desktop where the network of vm is set to NAT and one IP is assigned. You'll be able to connect to vnc service inside the docker image through that IP. Say it's `10.211.55.3` and we'll use it later.
 
 3. Setup the virtual machine for nested virtualization support
@@ -279,11 +308,11 @@ The following instructions are used for OS X. You'll need [docker-machine-parall
     ```
 
     If you use VMWare Fusion, go to menu bar > Vitual Machine > Settings > Processors and Memory, expand Advanced options, and select `Enable hypervisor applications in this virtual machine`.
-    
+
     ![Enable nested virtualization for VMWare Fusion](images/vmwarefusion_enable_nested_virtualization.png)
-    
+
     If you use Parallels Desktop, open settings screen of that vm and go to `CPU & Memory` under `hardware` tab, expand Advanced settings and select `Enable nested virtualization`.
-    
+
     ![Enable nested virtualization for Parallels Desktop](images/parallels_enable_nested_virtualization.png)
 
 4. Enable kvm inside virtual machine
@@ -340,6 +369,7 @@ Special Thanks
 [compose]: <images/compose.png>
 [line]: <https://github.com/butomo1989/docker-android/blob/master/docker-compose.yml#L70>
 [example of compose file]: <docker-compose.yml>
+[utils bash file]: <src/utils.sh>
 [docker-compose]: <https://docs.docker.com/compose/install/>
 [1.13.0]: <https://github.com/docker/compose/releases/tag/1.13.0>
 [Genymotion Cloud]: <https://www.genymotion.com/cloud/>

docker/Emulator_x86

@@ -89,20 +89,21 @@ ARG PROCESSOR=x86
 ARG SYS_IMG=x86
 ARG IMG_TYPE=google_apis
 ARG BROWSER=android
-ARG CHROME_DRIVER=24
+ARG CHROME_DRIVER=2.40
 ENV ANDROID_VERSION=$ANDROID_VERSION \
     API_LEVEL=$API_LEVEL \
     PROCESSOR=$PROCESSOR \
     SYS_IMG=$SYS_IMG \
     IMG_TYPE=$IMG_TYPE \
     BROWSER=$BROWSER \
-    CHROME_DRIVER=2.40
+    CHROME_DRIVER=$CHROME_DRIVER
 ENV PATH ${PATH}:${ANDROID_HOME}/build-tools
+
 RUN yes | sdkmanager --licenses && \
     sdkmanager "platforms;android-${API_LEVEL}" "system-images;android-${API_LEVEL};${IMG_TYPE};${SYS_IMG}" "emulator"
 
 #==============================================
-# Download latest version of chromedriver
+# Download proper version of chromedriver
 # to be able to use Chrome browser in emulator
 #==============================================
 RUN wget -nv -O chrome.zip "https://chromedriver.storage.googleapis.com/${CHROME_DRIVER}/chromedriver_linux64.zip" \
@@ -151,6 +152,6 @@ COPY supervisord.conf /root/
 RUN chmod -R +x /root/src && chmod +x /root/supervisord.conf
 
 HEALTHCHECK --interval=2s --timeout=40s --retries=1 \
-    CMD adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 1; done'
+    CMD timeout 40 adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 1; done'
 
 CMD /usr/bin/supervisord --configuration supervisord.conf

release.sh

@@ -31,6 +31,18 @@ declare -A list_of_levels=(
         [8.1]=27
 )
 
+# The version of the Chrome browser installed on the Android emulator needs to be known beforehand
+# in order to chose the proper version of chromedriver (see http://chromedriver.chromium.org/downloads)
+declare -A chromedriver_versions=(
+        [5.0.1]="2.12"
+        [5.1.1]="2.13"
+        [6.0]="2.18"
+        [7.0]="2.23"
+        [7.1.1]="2.28"
+        [8.0]="2.31"
+        [8.1]="2.33"
+)
+
 function get_android_versions() {
     versions=()
 
@@ -57,7 +69,6 @@ function get_android_versions() {
 
 get_android_versions
 processor=x86
-chrome_driver=$(curl -s https://chromedriver.storage.googleapis.com/LATEST_RELEASE)
 
 function test() {
     # Prepare needed parameter to run tests
@@ -141,6 +152,8 @@ function build() {
         echo "[BUILD] API Level: $level"
         sys_img=$processor
         echo "[BUILD] System Image: $sys_img"
+        chrome_driver="${chromedriver_versions[$v]}"
+        echo "[BUILD] chromedriver version: $chrome_driver"
         image_version="$IMAGE-$processor-$v:$RELEASE"
         image_latest="$IMAGE-$processor-$v:latest"
         echo "[BUILD] Image name: $image_version and $image_latest"

src/app.py

@@ -4,6 +4,7 @@ import json
 import logging
 import os
 import subprocess
+import uuid
 
 from src import CHROME_DRIVER, CONFIG_FILE, ROOT
 from src import log
@@ -49,12 +50,22 @@ def convert_str_to_bool(str: str) -> bool:
         logger.error(err)
 
 
+def is_initialized() -> bool:
+    return os.path.exists(INIT_FILE)
+
+
+def finish_initialization():
+    file = open(INIT_FILE, 'w+')
+    file.close()
+
+
 ANDROID_HOME = get_or_raise('ANDROID_HOME')
 ANDROID_VERSION = get_or_raise('ANDROID_VERSION')
 API_LEVEL = get_or_raise('API_LEVEL')
 PROCESSOR = get_or_raise('PROCESSOR')
 SYS_IMG = get_or_raise('SYS_IMG')
 IMG_TYPE = get_or_raise('IMG_TYPE')
+INIT_FILE = os.getenv('INIT_FILE', "/root/init")
 
 logger.info('Android version: {version} \n'
             'API level: {level} \n'
@@ -86,9 +97,10 @@ def prepare_avd(device: str, avd_name: str):
 
     avd_path = '/'.join([ANDROID_HOME, 'android_emulator'])
     creation_cmd = 'avdmanager create avd -f -n {name} -b {img_type}/{sys_img} -k "system-images;android-{api_lvl};' \
-        '{img_type};{sys_img}" -d {device} -p {path}'.format(name=avd_name, img_type=IMG_TYPE, sys_img=SYS_IMG,
-                                                             api_lvl=API_LEVEL, device=device_name_bash,
-                                                             path=avd_path)
+                   '{img_type};{sys_img}" -d {device} -p {path}'.format(name=avd_name, img_type=IMG_TYPE,
+                                                                        sys_img=SYS_IMG,
+                                                                        api_lvl=API_LEVEL, device=device_name_bash,
+                                                                        path=avd_path)
     logger.info('Command to create avd: {command}'.format(command=creation_cmd))
     subprocess.check_call(creation_cmd, shell=True)
 
@@ -114,8 +126,7 @@ def appium_run(avd_name: str):
         cmd += ' --relaxed-security'
 
     default_web_browser = os.getenv('BROWSER')
-    if default_web_browser == 'chrome':
-        cmd += ' --chromedriver-executable {driver}'.format(driver=CHROME_DRIVER)
+    cmd += ' --chromedriver-executable {driver}'.format(driver=CHROME_DRIVER)
 
     grid_connect = convert_str_to_bool(str(os.getenv('CONNECT_TO_GRID', False)))
     logger.info('Connect to selenium grid? {connect}'.format(connect=grid_connect))
@@ -182,18 +193,27 @@ def run():
     """Run app."""
     device = os.getenv('DEVICE', 'Nexus 5')
     logger.info('Device: {device}'.format(device=device))
+    device_id = os.getenv('DEVICE_ID', str(uuid.uuid4()))
+    logger.info('Device Id: {device_id}'.format(device_id=device_id))
 
     avd_name = '{device}_{version}'.format(device=device.replace(' ', '_').lower(), version=ANDROID_VERSION)
     logger.info('AVD name: {avd}'.format(avd=avd_name))
+    is_first_run = not is_initialized()
 
-    logger.info('Preparing emulator...')
-    prepare_avd(device, avd_name)
+    if is_first_run:
+        logger.info('Preparing emulator...')
+        prepare_avd(device, avd_name)
+        finish_initialization()
 
     logger.info('Run emulator...')
     dp_size = os.getenv('DATAPARTITION', '550m')
     with open("/root/android_emulator/config.ini", "a") as cfg:
         cfg.write('\ndisk.dataPartition.size={dp}'.format(dp=dp_size))
-    cmd = 'emulator/emulator @{name} -gpu off -verbose'.format(name=avd_name)
+
+    if is_first_run:
+        cmd = 'emulator/emulator @{name} -gpu off -prop emu.uuid={uuid} -verbose -wipe-data'.format(name=avd_name, uuid=device_id)
+    else:
+        cmd = 'emulator/emulator @{name} -gpu off -prop emu.uuid={uuid} -verbose'.format(name=avd_name, uuid=device_id)
     appium = convert_str_to_bool(str(os.getenv('APPIUM', False)))
     if appium:
         subprocess.Popen(cmd.split())
@@ -202,5 +222,6 @@ def run():
     else:
         result = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE).communicate()
 
+
 if __name__ == '__main__':
     run()

src/tests/unit/test_app.py

@@ -67,7 +67,7 @@ class TestApp(TestCase):
     @mock.patch('src.app.prepare_avd')
     @mock.patch('builtins.open')
     @mock.patch('subprocess.Popen')
-    def test_run_withhout_appium(self, mocked_avd, mocked_open, mocked_subprocess):
+    def test_run_without_appium(self, mocked_avd, mocked_open, mocked_subprocess):
         with mock.patch('src.app.appium_run') as mocked_appium:
             os.environ['APPIUM'] = str(False)
             app.run()
@@ -75,3 +75,21 @@ class TestApp(TestCase):
             self.assertTrue(mocked_open.called)
             self.assertTrue(mocked_subprocess.called)
             self.assertFalse(mocked_appium.called)
+
+    @mock.patch('builtins.open')
+    @mock.patch('subprocess.Popen')
+    @mock.patch('os.path.exists', mock.MagicMock(return_value=False))
+    def test_run_first_run(self, mocked_open, mocked_subprocess):
+        with mock.patch('src.app.prepare_avd') as mocked_prepare_avd:
+            app.run()
+            self.assertFalse(app.is_initialized())
+            self.assertTrue(mocked_prepare_avd.called)
+
+    @mock.patch('builtins.open')
+    @mock.patch('subprocess.Popen')
+    @mock.patch('os.path.exists', mock.MagicMock(return_value=True))
+    def test_run_next_run(self, mocked_open, mocked_subprocess):
+        with mock.patch('src.app.prepare_avd') as mocked_prepare_avd:
+            app.run()
+            self.assertTrue(app.is_initialized())
+            self.assertFalse(mocked_prepare_avd.called)

src/utils.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+BOOT ()
+{
+  A=$(adb wait-for-device shell getprop sys.boot_completed | tr -d '\r')
+  while [[ $A != "1" ]]; do
+          sleep 1;
+          A=$(adb wait-for-device shell getprop sys.boot_completed | tr -d '\r')
+  done;
+}
+
+Get_Google_Play_Services ()
+{
+  wget "https://www.apklinker.com/wp-content/uploads/uploaded_apk/5b51570a214a8/com.google.android.gms_12.8.74-040700-204998136_12874026_MinAPI23_(x86)(nodpi)_apklinker.com.apk"
+}
+
+Update_Google_Play_Services ()
+{
+  adb install -r "$PWD/com.google.android.gms_12.8.74-040700-204998136_12874026_MinAPI23_(x86)(nodpi)_apklinker.com.apk"
+}
+Disable_animations ()
+{
+  # this is for demonstration what other amazing staff can be done here
+  adb shell "settings put global window_animation_scale 0.0"
+  adb shell "settings put global transition_animation_scale 0.0"
+  adb shell "settings put global animator_duration_scale 0.0"
+}
+BOOT
+Get_Google_Play_Services
+Update_Google_Play_Services
+Disable_animations

supervisord.conf

@@ -55,3 +55,10 @@ autorestart=false
 stdout_logfile=%(ENV_LOG_PATH)s/video-recording.stdout.log
 stderr_logfile=%(ENV_LOG_PATH)s/video-recording.stderr.log
 priority=4
+
+[program:adb-utils]
+command=./src/utils.sh
+autorestart=false
+stdout_logfile=%(ENV_LOG_PATH)s/adb-utils.stdout.log
+stderr_logfile=%(ENV_LOG_PATH)s/adb-utils.stderr.log
+priority=5