Merge pull request #110 from devilankur18/emulator_backup_restore

AVD initialisation based on devices xml file

README.md

@@ -41,17 +41,17 @@ Advantages compare with other docker-android projects
 List of Docker images
 ---------------------
 
-|Supported OS   |Android version   |API level   |Image name   |Image status   |
+|OS   |Android   |API   |Browser   |Browser version   |Chromedriver   |Image   |Size   |
-|:---|:---|:---|:---|:---:|
+|:---|:---|:---|:---|:---|:---|:---|:---|
-|Linux|5.0.1|21|butomo1989/docker-android-x86-5.0.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.0.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.0.1 "Get your own image badge on microbadger.com")|
+|Linux|5.0.1|21|browser|37.0|2.12|butomo1989/docker-android-x86-5.0.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.0.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.0.1 "Get your own image badge on microbadger.com")|
-|Linux|5.1.1|22|butomo1989/docker-android-x86-5.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.1.1 "Get your own image badge on microbadger.com")|
+|Linux|5.1.1|22|browser|39.0|2.13|butomo1989/docker-android-x86-5.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-5.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-5.1.1 "Get your own image badge on microbadger.com")|
-|Linux|6.0|23|butomo1989/docker-android-x86-6.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-6.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-6.0 "Get your own image badge on microbadger.com")|
+|Linux|6.0|23|browser|44.0|2.18|butomo1989/docker-android-x86-6.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-6.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-6.0 "Get your own image badge on microbadger.com")|
-|Linux|7.0|24|butomo1989/docker-android-x86-7.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.0 "Get your own image badge on microbadger.com")|
+|Linux|7.0|24|chrome|51.0|2.23|butomo1989/docker-android-x86-7.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.0 "Get your own image badge on microbadger.com")|
-|Linux|7.1.1|25|butomo1989/docker-android-x86-7.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.1.1 "Get your own image badge on microbadger.com")|
+|Linux|7.1.1|25|chrome|55.0|2.28|butomo1989/docker-android-x86-7.1.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-7.1.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-7.1.1 "Get your own image badge on microbadger.com")|
-|Linux|8.0|26|butomo1989/docker-android-x86-8.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-8.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-8.0 "Get your own image badge on microbadger.com")|
+|Linux|8.0|26|chrome|58.0|2.31|butomo1989/docker-android-x86-8.0|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-8.0.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-8.0 "Get your own image badge on microbadger.com")|
-|Linux|8.1|27|butomo1989/docker-android-x86-8.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-8.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-8.1 "Get your own image badge on microbadger.com")|
+|Linux|8.1|27|chrome|61.0|2.33|butomo1989/docker-android-x86-8.1|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-x86-8.1.svg)](https://microbadger.com/images/butomo1989/docker-android-x86-8.1 "Get your own image badge on microbadger.com")|
-|All |-|-|butomo1989/docker-android-real-device|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-real-device.svg)](https://microbadger.com/images/butomo1989/docker-android-real-device "Get your own image badge on microbadger.com")|
+|All |-|-|-|-|-|butomo1989/docker-android-real-device|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-real-device.svg)](https://microbadger.com/images/butomo1989/docker-android-real-device "Get your own image badge on microbadger.com")|
-|Linux|All|All|butomo1989/docker-android-genymotion|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-genymotion.svg)](https://microbadger.com/images/butomo1989/docker-android-genymotion "Get your own image badge on microbadger.com")|
+|Linux|All|All|All|All|All|butomo1989/docker-android-genymotion|[![](https://images.microbadger.com/badges/image/butomo1989/docker-android-genymotion.svg)](https://microbadger.com/images/butomo1989/docker-android-genymotion "Get your own image badge on microbadger.com")|
 
 List of Devices
 ---------------
@@ -123,6 +123,20 @@ To run tests for mobile browser, following parameter can be passed:
 docker run --privileged -d -p 6080:6080 -p 4723:4723 -p 5554:5554 -p 5555:5555 -e DEVICE="Samsung Galaxy S6" -e APPIUM=true -e CONNECT_TO_GRID=true -e APPIUM_HOST="127.0.0.1" -e APPIUM_PORT=4723 -e SELENIUM_HOST="172.17.0.1" -e SELENIUM_PORT=4444 -e MOBILE_WEB_TEST=true --name android-container butomo1989/docker-android-x86-8.1
 ```
 
+### Back & Restore
+If you want to backup/reuse the avds created with furture upgrades or for replication, run the container with two extra mounts
+
+- -v local_backup/.android:/root/.android
+- -v local_backup/android_emulator:/root/android_emulator
+
+```bash
+docker run --privileged -d -p 6080:6080 -p 4723:4723 -p 5554:5554 -p 5555:5555 -v local_backup/.android:/root/.android -v local_backup/android_emulator:local_backup/android_emulator -e DEVICE="Nexus 5" -e APPIUM=true -e CONNECT_TO_GRID=true -e APPIUM_HOST="127.0.0.1" -e APPIUM_PORT=4723 -e SELENIUM_HOST="172.17.0.1" -e SELENIUM_PORT=4444 --name android-container butomo1989/docker-android-x86-8.1
+```
+For the first run, this will create a new avd and all the changes will be accessible in the `local_backup` directory. 
+
+Now for all future runs, it will reuse the avds. Even this should work with new releases of `docker-android`
+
+
 ### Share Volume
 
 If you want to use appium to test UI of your android application, you need to share volume where the APK is located to folder ***/root/tmp***.
@@ -135,6 +149,13 @@ docker run --privileged -d -p 6080:6080 -p 4723:4723 -p 5554:5554 -p 5555:5555 -
 
 You can deactivate auto_record by changing the value to "False" in docker-compose file. e.g. change value to "False" in this [line].
 
+### Language
+
+You can change the language setting of Android Emulator on the fly by passing following environment variable:
+
+- LANGUAGE="\<language>"
+- COUNTRY="\<country>"
+
 ### Docker-Compose
 
 ![][compose]
@@ -145,6 +166,10 @@ There is [example of compose file] to run complete selenium grid and docker-andr
 docker-compose up -d
 ```
 
+### Google Play Services and Google Play Store
+
+The Docker "Android Emulator" images contain Google Play Service (v12.8.74) and Google Play Store (v11.0.50). Both applications are downloaded from [apklinker](https://www.apklinker.com/), so please be aware of it in case you use private/company account to that applications.
+
 Build Android project
 ---------------------
 
@@ -167,7 +192,7 @@ Proxy
 
 You can enable proxy inside container by passing following environment variables:
 
-- HTTP_PROXY="\<docker\_bridge\_ip\_address>"
+- HTTP_PROXY="\<language>docker\_bridge\_ip\_address>"
 - HTTPS_PROXY="\<docker\_bridge\_ip\_address>"
 - NO_PROXY="localhost"
 
@@ -212,6 +237,15 @@ adb connect <docker-machine-ip-address>:5555
 
 **Note:** You need to have Android Debug Bridge (adb) installed in your host machine.
 
+Get rid of unauthorized device popup while connecting to the emulator outside the container
+------------------------------------------
+
+In order to connect to the emulator using already generated adbkey on your machine you will need to mount your `.android` folder to the container (this is needed only when you work with emulator based on playstore image):
+
+```
+docker run -v <local Android key directory>:/root/.android/ ...
+```
+
 SMS Simulation
 --------------
 
@@ -342,6 +376,7 @@ Special Thanks
 [compose]: <images/compose.png>
 [line]: <https://github.com/butomo1989/docker-android/blob/master/docker-compose.yml#L70>
 [example of compose file]: <docker-compose.yml>
+[utils bash file]: <src/utils.sh>
 [docker-compose]: <https://docs.docker.com/compose/install/>
 [1.13.0]: <https://github.com/docker/compose/releases/tag/1.13.0>
 [Genymotion Cloud]: <https://www.genymotion.com/cloud/>

docker/Emulator_x86

@@ -89,26 +89,37 @@ ARG PROCESSOR=x86
 ARG SYS_IMG=x86
 ARG IMG_TYPE=google_apis
 ARG BROWSER=android
-ARG CHROME_DRIVER=24
+ARG CHROME_DRIVER=2.40
+ARG GOOGLE_PLAY_SERVICE=12.8.74
+ARG GOOGLE_PLAY_STORE=11.0.50
 ENV ANDROID_VERSION=$ANDROID_VERSION \
     API_LEVEL=$API_LEVEL \
     PROCESSOR=$PROCESSOR \
     SYS_IMG=$SYS_IMG \
     IMG_TYPE=$IMG_TYPE \
     BROWSER=$BROWSER \
-    CHROME_DRIVER=2.40
+    CHROME_DRIVER=$CHROME_DRIVER \
+    GOOGLE_PLAY_SERVICE=$GOOGLE_PLAY_SERVICE \
+    GOOGLE_PLAY_STORE=$GOOGLE_PLAY_STORE
 ENV PATH ${PATH}:${ANDROID_HOME}/build-tools
+
 RUN yes | sdkmanager --licenses && \
     sdkmanager "platforms;android-${API_LEVEL}" "system-images;android-${API_LEVEL};${IMG_TYPE};${SYS_IMG}" "emulator"
 
 #==============================================
-# Download latest version of chromedriver
+# Download proper version of chromedriver
 # to be able to use Chrome browser in emulator
 #==============================================
 RUN wget -nv -O chrome.zip "https://chromedriver.storage.googleapis.com/${CHROME_DRIVER}/chromedriver_linux64.zip" \
  && unzip -x chrome.zip \
  && rm chrome.zip
 
+#================================================================
+# Download Google Play Services APK and Play Store from apklinker
+#================================================================
+Run wget -nv -O google_play_services.apk "https://www.apklinker.com/wp-content/uploads/uploaded_apk/5b5155e5ef4f8/com.google.android.gms_${GOOGLE_PLAY_SERVICE}-020700-204998136_12874013_MinAPI21_(x86)(nodpi)_apklinker.com.apk"
+Run wget -nv -O google_play_store.apk "https://www.apklinker.com/wp-content/uploads/uploaded_apk/5b632b1164e31/com.android.vending_${GOOGLE_PLAY_STORE}-all-0-PR-206665793_81105000_MinAPI16_(armeabi,armeabi-v7a,mips,mips64,x86,x86_64)(240,320,480dpi)_apklinker.com.apk"
+
 #================================================
 # noVNC Default Configurations
 # These Configurations can be changed through -e

release.sh

@@ -31,6 +31,18 @@ declare -A list_of_levels=(
         [8.1]=27
 )
 
+# The version of the Chrome browser installed on the Android emulator needs to be known beforehand
+# in order to chose the proper version of chromedriver (see http://chromedriver.chromium.org/downloads)
+declare -A chromedriver_versions=(
+        [5.0.1]="2.12"
+        [5.1.1]="2.13"
+        [6.0]="2.18"
+        [7.0]="2.23"
+        [7.1.1]="2.28"
+        [8.0]="2.31"
+        [8.1]="2.33"
+)
+
 function get_android_versions() {
     versions=()
 
@@ -57,9 +69,6 @@ function get_android_versions() {
 
 get_android_versions
 processor=x86
-#chrome_driver=$(curl -s https://chromedriver.storage.googleapis.com/LATEST_RELEASE)
-#Reason: https://sites.google.com/a/chromium.org/chromedriver/downloads
-chrome_driver=2.33
 
 function test() {
     # Prepare needed parameter to run tests
@@ -135,7 +144,8 @@ function build() {
             IMG_TYPE=google_apis
             BROWSER=browser
         else
-            IMG_TYPE=google_apis_playstore
+            #adb root cannot be run in IMG_TYPE=google_apis_playstore 
+            IMG_TYPE=google_apis
             BROWSER=chrome
         fi
         echo "[BUILD] IMAGE TYPE: $IMG_TYPE"
@@ -143,6 +153,8 @@ function build() {
         echo "[BUILD] API Level: $level"
         sys_img=$processor
         echo "[BUILD] System Image: $sys_img"
+        chrome_driver="${chromedriver_versions[$v]}"
+        echo "[BUILD] chromedriver version: $chrome_driver"
         image_version="$IMAGE-$processor-$v:$RELEASE"
         image_latest="$IMAGE-$processor-$v:latest"
         echo "[BUILD] Image name: $image_version and $image_latest"

src/app.py

@@ -4,6 +4,7 @@ import json
 import logging
 import os
 import subprocess
+import uuid
 
 from src import CHROME_DRIVER, CONFIG_FILE, ROOT
 from src import log
@@ -50,13 +51,7 @@ def convert_str_to_bool(str: str) -> bool:
 
 
 def is_initialized() -> bool:
-    return os.path.exists(INIT_FILE)
+    return os.path.exists(os.path.join(ROOT, '.android', 'devices.xml'))
-
-
-def finish_initialization():
-    file = open(INIT_FILE, 'w+')
-    file.close()
-
 
 ANDROID_HOME = get_or_raise('ANDROID_HOME')
 ANDROID_VERSION = get_or_raise('ANDROID_VERSION')
@@ -64,7 +59,6 @@ API_LEVEL = get_or_raise('API_LEVEL')
 PROCESSOR = get_or_raise('PROCESSOR')
 SYS_IMG = get_or_raise('SYS_IMG')
 IMG_TYPE = get_or_raise('IMG_TYPE')
-INIT_FILE = os.getenv('INIT_FILE', "/root/init")
 
 logger.info('Android version: {version} \n'
             'API level: {level} \n'
@@ -125,8 +119,7 @@ def appium_run(avd_name: str):
         cmd += ' --relaxed-security'
 
     default_web_browser = os.getenv('BROWSER')
-    if default_web_browser == 'chrome':
+    cmd += ' --chromedriver-executable {driver}'.format(driver=CHROME_DRIVER)
-        cmd += ' --chromedriver-executable {driver}'.format(driver=CHROME_DRIVER)
 
     grid_connect = convert_str_to_bool(str(os.getenv('CONNECT_TO_GRID', False)))
     logger.info('Connect to selenium grid? {connect}'.format(connect=grid_connect))
@@ -193,24 +186,26 @@ def run():
     """Run app."""
     device = os.getenv('DEVICE', 'Nexus 5')
     logger.info('Device: {device}'.format(device=device))
+    device_id = os.getenv('DEVICE_ID', str(uuid.uuid4()))
+    logger.info('Device Id: {device_id}'.format(device_id=device_id))
 
     avd_name = '{device}_{version}'.format(device=device.replace(' ', '_').lower(), version=ANDROID_VERSION)
     logger.info('AVD name: {avd}'.format(avd=avd_name))
+    is_first_run = not is_initialized()
 
-    if not is_initialized():
+    if is_first_run:
         logger.info('Preparing emulator...')
         prepare_avd(device, avd_name)
-        finish_initialization()
 
     logger.info('Run emulator...')
     dp_size = os.getenv('DATAPARTITION', '550m')
     with open("/root/android_emulator/config.ini", "a") as cfg:
         cfg.write('\ndisk.dataPartition.size={dp}'.format(dp=dp_size))
 
-    if not is_initialized():
+    if is_first_run:
-        cmd = 'emulator/emulator @{name} -gpu off -verbose -wipe-data'.format(name=avd_name)
+        cmd = 'emulator/emulator @{name} -gpu off -prop emu.uuid={uuid} -verbose -wipe-data'.format(name=avd_name, uuid=device_id)
     else:
-        cmd = 'emulator/emulator @{name} -gpu off -verbose'.format(name=avd_name)
+        cmd = 'emulator/emulator @{name} -gpu off -prop emu.uuid={uuid} -verbose'.format(name=avd_name, uuid=device_id)
     appium = convert_str_to_bool(str(os.getenv('APPIUM', False)))
     if appium:
         subprocess.Popen(cmd.split())

src/utils.sh

@@ -0,0 +1,44 @@
+#!/bin/bash
+
+function wait_emulator_to_be_ready () {
+  boot_completed=false
+  while [ "$boot_completed" == false ]; do
+    status=$(adb wait-for-device shell getprop sys.boot_completed | tr -d '\r')
+    echo "Boot Status: $status"
+
+    if [ "$status" == "1" ]; then
+      boot_completed=true
+    else
+      sleep 1
+    fi      
+  done
+}
+
+function change_language_if_needed() {
+  if [ ! -z "${LANGUAGE// }" ] && [ ! -z "${COUNTRY// }" ]; then
+    wait_emulator_to_be_ready
+    echo "Languge will be changed to ${LANGUAGE}-${COUNTRY}"
+    adb root && adb shell "setprop persist.sys.language $LANGUAGE; setprop persist.sys.country $COUNTRY; stop; start"
+    echo "Language is changed!"
+  fi
+}
+
+function install_google_play () {
+  wait_emulator_to_be_ready
+  echo "Google Play Service will be installed"
+  adb install -r "/root/google_play_services.apk"
+  echo "Google Play Store will be installed"
+  adb install -r "/root/google_play_store.apk"
+}
+
+function disable_animation () {
+  # To improve performance
+  adb shell "settings put global window_animation_scale 0.0"
+  adb shell "settings put global transition_animation_scale 0.0"
+  adb shell "settings put global animator_duration_scale 0.0"
+}
+
+change_language_if_needed
+sleep 1
+install_google_play
+disable_animation

supervisord.conf

@@ -55,3 +55,10 @@ autorestart=false
 stdout_logfile=%(ENV_LOG_PATH)s/video-recording.stdout.log
 stderr_logfile=%(ENV_LOG_PATH)s/video-recording.stderr.log
 priority=4
+
+[program:adb-utils]
+command=./src/utils.sh
+autorestart=false
+stdout_logfile=%(ENV_LOG_PATH)s/adb-utils.stdout.log
+stderr_logfile=%(ENV_LOG_PATH)s/adb-utils.stderr.log
+priority=5