Merge branch 'master' of github.com:butomo1989/docker-android

Added documentation how to connect emulator to the host

Analytics.md

@@ -28,7 +28,7 @@ Docker-Android's analytics are sent throughout Docker-Android's execution to Goo
 Docker-Android's analytics are accessible to Docker-Android's current maintainers. Contact [@butomo1989](https://github.com/butomo1989) if you are a maintainer and need access.
 
 ## How?
-The code is viewable in [this lines](./src/appium.sh#L203-L225).
+The code is viewable in [this lines](./src/appium.sh#L225-L247).
 
 ## Opting out before starting Docker-Android
 Docker-Android analytics helps us, maintainers and leaving it on is appreciated. However, if you want to opt out and not send any information, you can do this by using passing environment variable GA=false to the Docker container.

LICENSE.md

@@ -1,3 +1,5 @@
+## License Information
+
 Copyright 2016 budi utomo
 
 This program is subject to the terms of the Apache License, Version 2.0 AND the following amendments on forks and data processing. Thus, this is a Custom Apache 2.0 License, NOT a dual-license model you may choose from.

README.md

@@ -10,10 +10,6 @@
 
 Docker-Android is a docker image built to be used for everything related to mobile website testing and Android project.
 
-<p align="center">
-   <a href="https://youtu.be/pQdpjuYwvp8"><img src="./images/appiumconf2018.png" alt="Appium Conference 2018" width="600"></a>
-</p>
-
 Emulator - Samsung Device    | Emulator - Nexus Device     | Real Device  
 :---------------------------:|:---------------------------:|:---------------------------:
 ![][emulator samsung]        |![][emulator nexus]          |![][real device]   
@@ -119,6 +115,33 @@ Docker-Android can be used for building Android project and executing its unit t
     ```bash
     docker run -it --rm -v $PWD/android-testing/ui/espresso/BasicSample:/root/tmp butomo1989/docker-android-x86-8.1 tmp/gradlew build
     ```
+    
+Control Android connected to host (Emulator or Real Device)
+-----------------------------------------------------------
+1. Create a docker container with this command 
+
+	```
+	$ docker run --privileged -d -p 6080:6080 -p 5554:5554 -p 5555:5555 -p 4723:4723 --name android-container-appium butomo1989/docker-android-real-device
+	```
+	
+2. Open noVNC [http://localhost:6080](http://localhost:6080)
+
+3. Open terminal by clicking right on **noVNC** window >> **Terminal emulator**
+
+4. To connect to host's adb (make sure your host have adb and connected to the device.) 
+
+	```
+	$ adb -H host.docker.internal devices
+	```
+	
+	To specify port, just add `-P port_number`
+
+	```
+	$ adb -H host.docker.internal -P 5037 devices
+	```
+	
+5. Now your container can access your host devices. But, you need to add `remoteAdbHost` and `adbPort` desired capabilities to make **Appium** can recognise those devices.  
+
 
 Appium and Selenium Grid
 ------------------------
@@ -192,9 +215,11 @@ This [document](README_CLOUD.md) contains information about deploying Docker-And
 Genymotion
 ----------
 
-![Genymotion](images/logo_genymotion.png)
+<p align="center">
+  <img id="geny" src="./images/logo_genymotion_and_dockerandroid.png" />
+</p>
 
-For you who do not have ressources to maintain the simulator or to buy machines or need different device profiles, you need to give a try to [Genymotion Cloud](https://www.genymotion.com/cloud/). Docker-Android is integrated with Genymotion on different cloud services, e.g. Genymotion Cloud, AWS, GCP, Alibaba Cloud. Please follow [this document](README_GENYMOTION.md) for more detail.
+For you who do not have ressources to maintain the simulator or to buy machines or need different device profiles, you need to give a try to [Genymotion Cloud](https://www.genymotion.com/cloud/). Docker-Android is integrated with Genymotion on different cloud services, e.g. Genymotion Cloud, AWS, GCP, Alibaba Cloud. Please follow [this document](README_GENYMOTION.md) or [this blog](https://medium.com/genymobile/run-your-appium-tests-using-docker-android-genymotion-cloud-e4817132ccd8) for more detail.
 
 Troubleshooting
 ---------------

README_CUSTOM_CONFIG.md

@@ -3,7 +3,7 @@ Proxy
 
 You can enable proxy inside container by passing following environment variables:
 
-- HTTP_PROXY="\<language>docker\_bridge\_ip\_address>"
+- HTTP_PROXY="\<docker\_bridge\_ip\_address>"
 - HTTPS_PROXY="\<docker\_bridge\_ip\_address>"
 - NO_PROXY="localhost"
 

README_GENYMOTION.md

@@ -1,6 +1,8 @@
 Genymotion Cloud
 ----------------
 
+![Genymotion](images/logo_genymotion.png)
+
 You can easily scale your Appium tests on Genymotion Android virtual devices in the cloud. They are available on SaaS or as virtual images on AWS, GCP or Alibaba Cloud.
 
 1. On SaaS <br />
@@ -11,7 +13,7 @@ You can easily scale your Appium tests on Genymotion Android virtual devices in
 	export PASS="xxx"
 	export LICENSE="xxx"
 
-	docker run -it --rm -p 4723:4723 -v $PWD/genymotion/example/sample_devices:/root/tmp -e TYPE=genycloud -e USER=$USER -e PASS=$PASS -e LICENSE=$LICENSE butomo1989/docker-android-genymotion
+	docker run -it --rm -p 4723:4723 -v $PWD/genymotion/example/sample_devices:/root/tmp -e TYPE=SaaS -e USER=$USER -e PASS=$PASS -e LICENSE=$LICENSE butomo1989/docker-android-genymotion
 	```
 
 <p align="center">

docker/Emulator_x86

@@ -1,4 +1,4 @@
-FROM appium/appium:1.9.1-p0
+FROM appium/appium:1.10.1-p0
 
 LABEL maintainer "Budi Utomo <budi.ut.1989@gmail.com>"
 

docker/Genymotion

@@ -1,4 +1,4 @@
-FROM appium/appium:1.9.1-p0
+FROM appium/appium:1.10.1-p0
 
 LABEL maintainer "Budi Utomo <budi.ut.1989@gmail.com>"
 

docker/Real_device

@@ -1,4 +1,4 @@
-FROM appium/appium:1.9.1-p0
+FROM appium/appium:1.10.1-p0
 
 LABEL maintainer "Budi Utomo <budi.ut.1989@gmail.com>"
 

genymotion/example/geny.yml

@@ -11,7 +11,7 @@ services:
       - 4444:4444
 
   # Please stop this container by using docker stop instead of docker-compose stop
-  cloud:
+  saas:
     image: butomo1989/docker-android-genymotion
     depends_on:
       - selenium_hub
@@ -20,7 +20,7 @@ services:
       - $PWD/sample_apk:/root/tmp/sample_apk
       - $PWD/sample_devices:/root/tmp
     environment:
-      - TYPE=genycloud
+      - TYPE=SaaS
       - TEMPLATE=/root/tmp/devices.json
       - USER=$USER
       - PASS=$PASS

genymotion/example/sample_devices/aws.json

@@ -1,12 +1,60 @@
 [
-	{
-		"region": "eu-west-1",
-		"android_version": "6.0",
-		"instance": "t2.small"
-	},
 	{
 		"region": "eu-west-1",
 		"android_version": "8.0",
 		"instance": "t2.small"
+	},
+	{
+		"region": "eu-west-1",
+		"instance": "t2.small",
+		"AMI": "ami-68d78411",
+		"SG": [
+			{
+				"type": "ingress",
+				"configurations": [
+					{
+						"from_port": 22,
+						"to_port": 22,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 80,
+						"to_port": 80,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 443,
+						"to_port": 443,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 51000,
+						"to_port": 51100,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 51000,
+						"to_port": 51100,
+						"protocol": "udp",
+						"cidr_blocks": "0.0.0.0/0"
+					}
+				]
+			},
+			{
+				"type": "egress",
+				"configurations": [
+					{
+						"from_port": 0,
+						"to_port": 65535,
+						"protocol": "udp",
+						"cidr_blocks": "0.0.0.0/0"
+					}
+				]
+			}
+		]
 	}
 ]

genymotion/geny_start.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 # This script is needed because of https://www.ctl.io/developers/blog/post/gracefully-stopping-docker-containers/
 
-types=(genycloud aws)
+types=(saas aws)
 
 if [ -z "$TYPE" ]; then
     echo "Please specify one of following types: ${types[@]}"

src/app.py

@@ -124,7 +124,8 @@ def appium_run(avd_name: str):
     grid_connect = convert_str_to_bool(str(os.getenv('CONNECT_TO_GRID', False)))
     logger.info('Connect to selenium grid? {connect}'.format(connect=grid_connect))
     if grid_connect:
-        local_ip = os.popen('ifconfig eth0 | grep \'inet addr:\' | cut -d: -f2 | awk \'{ print $1}\'').read().strip()
+        # Ubuntu 16.04 -> local_ip = os.popen('ifconfig eth0 | grep \'inet addr:\' | cut -d: -f2 | awk \'{ print $1}\'').read().strip()
+        local_ip = os.popen('ifconfig eth0 | grep \'inet\' | cut -d: -f2 | awk \'{ print $2}\'').read().strip()
         try:
             mobile_web_test = convert_str_to_bool(str(os.getenv('MOBILE_WEB_TEST', False)))
             appium_host = os.getenv('APPIUM_HOST', local_ip)

src/appium.sh

@@ -46,13 +46,118 @@ function prepare_geny_aws() {
 			echo ${row} | base64 --decode | jq -r ${1}
 		}
 
-	    region=$(get_value '.region')
-	    android_version=$(get_value '.android_version')
-	    instance=$(get_value '.instance')
+		region=$(get_value '.region')
+		android_version=$(get_value '.android_version')
+		instance=$(get_value '.instance')
+		ami=$(get_value '.AMI')
+		sg=$(get_value '.SG')
 
-	    echo $region
-	    echo $android_version
-	    echo $instance
+		echo $region
+		echo $android_version
+		echo $instance
+		echo $ami
+		echo $sg
+
+	    #TODO: remove this dirty hack
+		if [[ $android_version == null ]]; then
+			echo "[HACK] Version cannot be empty! version will be added!"
+			android_version="6.0"
+		fi
+
+		#Custom Security Group
+		if [[ $sg != null ]]; then
+			echo "Custom security group is found!"
+			security_group=""
+
+			is_array=$(echo "${sg}" | jq 'if type=="array" then true else false end')
+			if [ $is_array == "true" ]; then
+				echo "New security group with given rules will be created"
+				for i in $(echo "${sg}" | jq -r '.[] | @base64'); do
+					get_value() {
+						echo ${i} | base64 --decode | jq -r ${1}
+					}
+
+					type=$(get_value '.type')
+					configs=$(get_value '.configurations')
+
+
+					for c in $(echo "${configs}" | jq -r '.[] | @base64'); do
+						get_value() {
+							echo ${c} | base64 --decode | jq -r ${1}
+						}
+
+						from_port=$(get_value '.from_port')
+						to_port=$(get_value '.to_port')
+						protocol=$(get_value '.protocol')
+						cidr_blocks=$(get_value '.cidr_blocks')
+						security_group+=$(cat <<_EOF
+
+	$type {
+		from_port	= $from_port
+		to_port		= $to_port
+		protocol	= "$protocol"
+		cidr_blocks	= ["$cidr_blocks"]
+	}
+_EOF
+	)
+					done
+				done
+			else
+				#TODO: remove this dirty hack
+				echo "Given security group will be used!"
+				is_array="false"
+				security_group=$(cat <<_EOF
+	ingress {
+		from_port       = 22
+		to_port         = 22
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+_EOF
+)
+			fi
+		else
+			echo "Custom security is not found! It will use default security group!"
+			security_group=$(cat <<_EOF
+	ingress {
+		from_port       = 22
+		to_port         = 22
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 80
+		to_port         = 80
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 443
+		to_port         = 443
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 51000
+		to_port         = 51100
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 51000
+		to_port         = 51100
+		protocol        = "udp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	egress {
+		from_port       = 0
+		to_port         = 65535
+		protocol        = "udp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+_EOF
+)
+		fi
 
 	    aws_tf_content=$(cat <<_EOF
 variable "aws_region_$index" {
@@ -61,8 +166,8 @@ variable "aws_region_$index" {
 }
 
 variable "android_version_$index" {
-    type    = "string"
-    default = "$android_version"
+	type    = "string"
+	default = "$android_version"
 }
 
 variable "instance_type_$index" {
@@ -77,39 +182,7 @@ provider "aws" {
 
 resource "aws_security_group" "geny_sg_$index" {
 	provider = "aws.provider_$index"
-	ingress {
-		from_port       = 22
-		to_port         = 22
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-		description     = "SSH access"
-	}
-	ingress {
-		from_port       = 80
-		to_port         = 80
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-		description     = "HTTP access"
-	}
-	ingress {
-		from_port       = 443
-		to_port         = 443
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-		description     = "HTTPS access"
-	}
-	ingress {
-		from_port       = 51000
-		to_port         = 51100
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-	}
-	ingress {
-		from_port       = 51000
-		to_port         = 51100
-		protocol        = "udp"
-		cidr_blocks     = ["0.0.0.0/0"]
-	}
+	$security_group
 }
 
 data "aws_ami" "geny_aws_$index" {
@@ -131,9 +204,9 @@ resource "aws_key_pair" "geny_key_$index" {
 
 resource "aws_instance" "geny_aws_$index" {
 	provider      = "aws.provider_$index"
-	ami           = "\${data.aws_ami.geny_aws_$index.id}"
+	ami="\${data.aws_ami.geny_aws_$index.id}"
 	instance_type = "\${var.instance_type_$index}"
-	vpc_security_group_ids = ["\${aws_security_group.geny_sg_$index.name}"]
+	vpc_security_group_ids=["\${aws_security_group.geny_sg_$index.name}"]
 	key_name      = "\${aws_key_pair.geny_key_$index.key_name}"
 	tags {
 		Name = "DockerAndroid-\${data.aws_ami.geny_aws_$index.id}"
@@ -157,6 +230,21 @@ output "public_dns_$index" {
 _EOF
 )
 		echo "$aws_tf_content" > /root/aws_tf_$index.tf
+
+		if [[ $ami != null ]]; then
+			echo "Using given AMI!"
+			sed -i "s/.*ami=.*/        ami=\"$ami\"/g" /root/aws_tf_$index.tf
+		else
+			echo "Custom AMI is not found. It will use the latest AMI!"
+		fi
+
+		if [[ $sg != null ]] && [[ $is_array == "false" ]]; then
+			echo "Using given security group: $sg"
+			sed -i "s/.*vpc_security_group_ids=.*/        vpc_security_group_ids=[\"$sg\"]/g" /root/aws_tf_$index.tf
+		fi
+
+		echo "---------------------------------------------------------"
+
 	    ((index++))
 	    ((port++))
 	done
@@ -173,7 +261,7 @@ _EOF
 	echo "Connect to adb remotely"
 	for ((i=index;i>=1;i--)); do	
 		dns=$(./terraform output public_dns_$i)	
-		((sleep ${interval_sleep} && adb connect localhost:${port}) > /dev/null & ssh -i ~/.ssh/id_rsa -oStrictHostKeyChecking=no -q -NL ${port}:localhost:5555 shell@${dns}) &
+		((sleep ${interval_sleep} && adb connect localhost:${port}) > /dev/null & ssh -i ~/.ssh/id_rsa -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -q -NL ${port}:localhost:5555 shell@${dns}) &
 		((port--))
 		time_sleep=$((time_sleep+interval_sleep))
 	done
@@ -203,7 +291,7 @@ function run_appium() {
 function ga(){
 	if [ "$GA" = true ]; then
 		echo "Collecting data for improving the project"
-		description="PROCESSOR: ${SYS_IMG}; DEVICE: ${DEVICE}; APPIUM: ${APPIUM}; SELENIUM: ${CONNECT_TO_GRID}; MOBILE_TEST: ${MOBILE_WEB_TEST}"
+		description="PROCESSOR: ${SYS_IMG}; VERSION: ${ANDROID_VERSION}; DEVICE: ${DEVICE}; APPIUM: ${APPIUM}; SELENIUM: ${CONNECT_TO_GRID}; MOBILE_TEST: ${MOBILE_WEB_TEST}"
 		random_user=$(cat /proc/version 2>&1 | sed -e 's/ /_/g' | sed -e 's/[()]//g' | sed -e 's/@.*_gcc_version/_gcc/g' | sed -e 's/__/_/g' | sed -e 's/Linux_version_//g' | sed -e 's/generic_build/genb/g')
 		random_user="${APP_RELEASE_VERSION}_${random_user}"
 		payload=(
@@ -224,6 +312,7 @@ function ga(){
 	fi
 }
 
+ga
 if [ "$REAL_DEVICE" = true ]; then
 	echo "Using real device"
 	run_appium
@@ -232,7 +321,7 @@ elif [ "$GENYMOTION" = true ]; then
 	echo "${types[@]}"
 	case $TYPE in
 	"${types[0]}" )
-		echo "Using Genymotion-Cloud"
+		echo "Using Genymotion-Cloud (SaaS)"
 		prepare_geny_cloud
 		run_appium
 		;;

src/port_forward.sh

@@ -1,5 +1,6 @@
 #!/bin/bash
 
-ip=$(ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}')
+#Ubuntu 16.04 -> ip=$(ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}')
+ip=$(ifconfig eth0 | grep 'inet' | cut -d: -f2 | awk '{ print $2}')
 socat tcp-listen:5554,bind=$ip,fork tcp:127.0.0.1:5554 &
 socat tcp-listen:5555,bind=$ip,fork tcp:127.0.0.1:5555

src/tests/unit/test_app.py

@@ -64,6 +64,19 @@ class TestApp(TestCase):
             self.assertTrue(mocked_subprocess.called)
             self.assertTrue(mocked_appium.called)
 
+    @mock.patch('src.app.prepare_avd')
+    @mock.patch('builtins.open')
+    @mock.patch('subprocess.Popen')
+    def test_run_with_appium_and_relaxed_security(self, mocked_avd, mocked_open, mocked_subprocess):
+        with mock.patch('src.app.appium_run') as mocked_appium:
+            os.environ['APPIUM'] = str(True)
+            os.environ['RELAXED_SECURITY'] = str(True)
+            app.run()
+            self.assertTrue(mocked_avd.called)
+            self.assertTrue(mocked_open.called)
+            self.assertTrue(mocked_subprocess.called)
+            self.assertTrue(mocked_appium.called)
+
     @mock.patch('src.app.prepare_avd')
     @mock.patch('builtins.open')
     @mock.patch('subprocess.Popen')