Merge branch 'master' of github.com:butomo1989/docker-android

.travis.yml

@@ -20,6 +20,7 @@ env:
   - ANDROID_VERSION=8.1
   - ANDROID_VERSION=9.0
   - REAL_DEVICE=True
+  - GENYMOTION=True
 
 script: bash travis.sh
 

Analytics.md

@@ -0,0 +1,37 @@
+# Docker-Android's Anonymous Aggregate User Behaviour Analytics
+Docker-Android has begun gathering anonymous aggregate user behaviour analytics and reporting these to Google Analytics. You are notified about this when you start Docker-Android.
+
+## Why?
+Docker-Android is provided free of charge for our internal and external users and we don't have direct communication with its users nor time resources to ask directly for their feedback. As a result, we now use anonymous aggregate user analytics to help us understand how Docker-Android is being used, the most common used features based on how, where and when people use it. With this information we can prioritize some features over other ones.
+
+## What?
+Docker-Android's analytics record some shared information for every event:
+
+- The Google Analytics version i.e. `1` (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#v)
+- The Google Analytics anonymous IP setting is enabled i.e. `1` (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#aip)
+- The Docker-Android analytics tracking ID e.g. `UA-128205228-1` (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#tid)
+- The release version of machine, e.g. `Linux_version_4.4.16-boot2docker_(gcc_version_4.9.2_(Debian_4.9.2-10)_)_#1_SMP_Fri_Jul_29_00:13:24_UTC_2016` This does not allow us to track individual users but does enable us to accurately measure user counts vs. event counts (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#cid)
+- Docker-Android analytics hit type, e.g. `event` (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#t)
+- Application type, e.g. `Emulator` (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#ec)
+- Description will contains information about Emulator configuration, e.g. `Processor type`. (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#el)
+- Docker-Android application name, e.g. `docker-android` (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#an)
+- Docker-Android application version, e.g. `1.5-p0` (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#av)
+
+With the recorded information, it is not possible for us to match any particular real user.
+
+As far as we can tell it would be impossible for Google to match the randomly generated analytics user ID to any other Google Analytics user ID. If Google turned evil the only thing they could do would be to lie about anonymising IP addresses and attempt to match users based on IP addresses.
+
+## When/Where?
+Docker-Android's analytics are sent throughout Docker-Android's execution to Google Analytics over HTTPS.
+
+## Who?
+Docker-Android's analytics are accessible to Docker-Android's current maintainers. Contact [@butomo1989](https://github.com/butomo1989) if you are a maintainer and need access.
+
+## How?
+The code is viewable in [this lines](./src/appium.sh#L225-L247).
+
+## Opting out before starting Docker-Android
+Docker-Android analytics helps us, maintainers and leaving it on is appreciated. However, if you want to opt out and not send any information, you can do this by using passing environment variable GA=false to the Docker container.
+
+## Disclaimer
+This document and the implementation are based on the great idea implemented by [Homebrew](https://github.com/Homebrew/brew/blob/master/docs/Analytics.md)

LICENSE

@@ -1,13 +0,0 @@
-Copyright 2016 budi utomo
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-  http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.

LICENSE.md

@@ -0,0 +1,36 @@
+## License Information
+
+Copyright 2016 budi utomo
+
+This program is subject to the terms of the Apache License, Version 2.0 AND the following amendments on forks and data processing. Thus, this is a Custom Apache 2.0 License, NOT a dual-license model you may choose from.
+
+
+You can obtain a copy of the license at http://www.apache.org/licenses/LICENSE-2.0
+
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
+
+
+## Forks
+Additionally to Apache-2.0, when you fork this repo you are required to either remove our Google Analytics tracking ID: UA-128205228-1 or stop usage gathering completely.
+
+## Data processing agreement
+By using this software you agree that the following non-PII (non personally identifiable information) data will be collected, processed and used by the maintainers for the purpose of improving the docker-android project. Anonymisation with respect of the IP address means that only the first two octets of the IP address are collected.
+
+
+By using this software you also grant us a nonexclusive, irrevocable, world-wide, perpetual royalty-free permission to use, modify and publish these data for all purposes, internally or publicly, including the right to sub-license said permission rights.
+
+
+We collect, process and use the following data:
+
+* Release version of Docker-Android
+* Anonymized IP address (only first two octets)
+* Country and city
+* Date and time when Docker-Android started
+* User (it will collect the information about Release Version of Machine)
+* Application type, e.g. Emulator or Device or Genymotion
+* Emulator configuration, e.g. Processor type, Device name, Appium mode, Selenium grid mode and mobile test mode
+
+## End of License Information
+
+More information about anonymized data collection can be seen [here](Analytics.md)

README.md

@@ -2,6 +2,7 @@
   <img id="header" src="./images/logo_dockerandroid_small.png" />
 </p>
 
+[![Analytics](https://ga-beacon.appspot.com/UA-128205228-1/github/butomo1989/docker-android/README.md)](https://github.com/igrigorik/ga-beacon "Analytics")
 [![Join the chat at https://gitter.im/butomo1989/docker-android](https://badges.gitter.im/butomo1989/docker-android.svg)](https://gitter.im/butomo1989/docker-android?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 [![Build Status](https://travis-ci.org/butomo1989/docker-android.svg?branch=master)](https://travis-ci.org/butomo1989/docker-android)
 [![codecov](https://codecov.io/gh/butomo1989/docker-android/branch/master/graph/badge.svg)](https://codecov.io/gh/butomo1989/docker-android)
@@ -207,6 +208,10 @@ Special Thanks
 --------------
 - [Gian Christanto] for creating a great logo!
 
+LICENSE
+--------------
+See [License](LICENSE.md)
+
 [appium]: <https://appium.io>
 [espresso]: <https://developer.android.com/training/testing/espresso/>
 [robotium]: <https://github.com/RobotiumTech/robotium>

README_CUSTOM_CONFIG.md

@@ -3,7 +3,7 @@ Proxy
 
 You can enable proxy inside container by passing following environment variables:
 
-- HTTP_PROXY="\<language>docker\_bridge\_ip\_address>"
+- HTTP_PROXY="\<docker\_bridge\_ip\_address>"
 - HTTPS_PROXY="\<docker\_bridge\_ip\_address>"
 - NO_PROXY="localhost"
 

docker/Emulator_x86

@@ -92,6 +92,7 @@ ARG BROWSER=android
 ARG CHROME_DRIVER=2.40
 ARG GOOGLE_PLAY_SERVICE=12.8.74
 ARG GOOGLE_PLAY_STORE=11.0.50
+ARG APP_RELEASE_VERSION=1.5-p0
 ENV ANDROID_VERSION=$ANDROID_VERSION \
     API_LEVEL=$API_LEVEL \
     PROCESSOR=$PROCESSOR \
@@ -100,7 +101,13 @@ ENV ANDROID_VERSION=$ANDROID_VERSION \
     BROWSER=$BROWSER \
     CHROME_DRIVER=$CHROME_DRIVER \
     GOOGLE_PLAY_SERVICE=$GOOGLE_PLAY_SERVICE \
-    GOOGLE_PLAY_STORE=$GOOGLE_PLAY_STORE
+    GOOGLE_PLAY_STORE=$GOOGLE_PLAY_STORE \
+    GA=true \
+    GA_ENDPOINT=https://www.google-analytics.com/collect \
+    GA_TRACKING_ID=UA-128205228-1 \
+    GA_API_VERSION="1" \
+    APP_RELEASE_VERSION=$APP_RELEASE_VERSION \
+    APP_TYPE=Emulator
 ENV PATH ${PATH}:${ANDROID_HOME}/build-tools
 
 RUN yes | sdkmanager --licenses && \

docker/Genymotion

@@ -80,6 +80,7 @@ RUN  wget -nv -O noVNC.zip "https://github.com/kanaka/noVNC/archive/${NOVNC_SHA}
 # noVNC Default Configurations
 # These Configurations can be changed through -e
 #================================================
+ARG APP_RELEASE_VERSION=1.5-p0
 ENV DISPLAY=:0 \
     SCREEN=0 \
     SCREEN_WIDTH=1600 \
@@ -89,7 +90,13 @@ ENV DISPLAY=:0 \
     TARGET_PORT=6080 \
     TIMEOUT=1 \
     VIDEO_PATH=/tmp/video \
-    LOG_PATH=/var/log/supervisor 
+    LOG_PATH=/var/log/supervisor \
+    GA=true \
+    GA_ENDPOINT=https://www.google-analytics.com/collect \
+    GA_TRACKING_ID=UA-128205228-1 \
+    GA_API_VERSION="1" \
+    APP_RELEASE_VERSION=$APP_RELEASE_VERSION \
+    APP_TYPE=Genymotion
 
 #====================
 # Install genymotion

docker/Real_device

@@ -71,6 +71,7 @@ RUN  wget -nv -O noVNC.zip "https://github.com/kanaka/noVNC/archive/${NOVNC_SHA}
 # noVNC Default Configurations
 # These Configurations can be changed through -e
 #================================================
+ARG APP_RELEASE_VERSION=1.5-p0
 ENV DISPLAY=:0 \
     SCREEN=0 \
     SCREEN_WIDTH=1600 \
@@ -80,7 +81,13 @@ ENV DISPLAY=:0 \
     TARGET_PORT=6080 \
     TIMEOUT=1 \
     VIDEO_PATH=/tmp/video \
-    LOG_PATH=/var/log/supervisor
+    LOG_PATH=/var/log/supervisor \
+    GA=true \
+    GA_ENDPOINT=https://www.google-analytics.com/collect \
+    GA_TRACKING_ID=UA-128205228-1 \
+    GA_API_VERSION="1" \
+    APP_RELEASE_VERSION=$APP_RELEASE_VERSION \
+    APP_TYPE=Device
 
 #=========================
 # Set default variables

genymotion/example/sample_devices/aws.json

@@ -1,12 +1,60 @@
 [
-	{
-		"region": "eu-west-1",
-		"android_version": "6.0",
-		"instance": "t2.small"
-	},
 	{
 		"region": "eu-west-1",
 		"android_version": "8.0",
 		"instance": "t2.small"
+	},
+	{
+		"region": "eu-west-1",
+		"instance": "t2.small",
+		"AMI": "ami-68d78411",
+		"SG": [
+			{
+				"type": "ingress",
+				"configurations": [
+					{
+						"from_port": 22,
+						"to_port": 22,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 80,
+						"to_port": 80,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 443,
+						"to_port": 443,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 51000,
+						"to_port": 51000,
+						"protocol": "tcp",
+						"cidr_blocks": "0.0.0.0/0"
+					},
+					{
+						"from_port": 51000,
+						"to_port": 51100,
+						"protocol": "udp",
+						"cidr_blocks": "0.0.0.0/0"
+					}
+				]
+			},
+			{
+				"type": "egress",
+				"configurations": [
+					{
+						"from_port": 0,
+						"to_port": 65535,
+						"protocol": "udp",
+						"cidr_blocks": "0.0.0.0/0"
+					}
+				]
+			}
+		]
 	}
 ]

release.sh

@@ -163,7 +163,8 @@ function build() {
         echo "[BUILD] Dockerfile: $FILE_NAME"
         docker build -t $image_version --build-arg ANDROID_VERSION=$v --build-arg API_LEVEL=$level \
         --build-arg PROCESSOR=$processor --build-arg SYS_IMG=$sys_img --build-arg IMG_TYPE=$IMG_TYPE \
-        --build-arg BROWSER=$BROWSER --build-arg CHROME_DRIVER=$chrome_driver -f $FILE_NAME .
+        --build-arg BROWSER=$BROWSER --build-arg CHROME_DRIVER=$chrome_driver \
+        --build-arg APP_RELEASE_VERSION=$RELEASE -f $FILE_NAME .
         docker tag $image_version $image_latest
     done
 }

release_geny.sh

@@ -0,0 +1,48 @@
+#!/bin/bash
+
+if [ -z "$1" ]; then
+    read -p "Task (build|push|all) : " TASK
+else
+    TASK=$1
+fi
+
+if [ -z "$2" ]; then
+    read -p "Release version: " RELEASE
+else
+    RELEASE=$2
+fi
+
+IMAGE="butomo1989/docker-android"
+FILE_NAME=docker/Genymotion
+
+image_version="$IMAGE-genymotion:$RELEASE"
+image_latest="$IMAGE-genymotion:latest"
+
+function build() {
+  echo "[BUILD] Image name: $image_version and $image_latest"
+  echo "[BUILD] Dockerfile: $FILE_NAME"
+  docker build -t $image_version --build-arg APP_RELEASE_VERSION=$RELEASE -f $FILE_NAME .
+  docker tag $image_version $image_latest
+}
+
+function push() {
+  echo "[PUSH] Image name: $image_version and $image_latest"
+  docker push $image_version
+  docker push $image_latest
+}
+
+case $TASK in
+    build)
+        build
+    ;;
+    push)
+        push
+    ;;
+    all)
+        build
+        push
+    ;;
+    *)
+        echo "Invalid environment! Valid options: test, build, push, all"
+    ;;
+esac

release_real.sh

@@ -21,7 +21,7 @@ image_latest="$IMAGE-real-device:latest"
 function build() {
   echo "[BUILD] Image name: $image_version and $image_latest"
   echo "[BUILD] Dockerfile: $FILE_NAME"
-  docker build -t $image_version -f $FILE_NAME .
+  docker build -t $image_version --build-arg APP_RELEASE_VERSION=$RELEASE -f $FILE_NAME .
   docker tag $image_version $image_latest
 }
 

src/appium.sh

@@ -46,13 +46,118 @@ function prepare_geny_aws() {
 			echo ${row} | base64 --decode | jq -r ${1}
 		}
 
-	    region=$(get_value '.region')
-	    android_version=$(get_value '.android_version')
-	    instance=$(get_value '.instance')
+		region=$(get_value '.region')
+		android_version=$(get_value '.android_version')
+		instance=$(get_value '.instance')
+		ami=$(get_value '.AMI')
+		sg=$(get_value '.SG')
 
-	    echo $region
-	    echo $android_version
-	    echo $instance
+		echo $region
+		echo $android_version
+		echo $instance
+		echo $ami
+		echo $sg
+
+	    #TODO: remove this dirty hack
+		if [[ $android_version == null ]]; then
+			echo "[HACK] Version cannot be empty! version will be added!"
+			android_version="6.0"
+		fi
+
+		#Custom Security Group
+		if [[ $sg != null ]]; then
+			echo "Custom security group is found!"
+			security_group=""
+
+			is_array=$(echo "${sg}" | jq 'if type=="array" then true else false end')
+			if [ $is_array == "true" ]; then
+				echo "New security group with given rules will be created"
+				for i in $(echo "${sg}" | jq -r '.[] | @base64'); do
+					get_value() {
+						echo ${i} | base64 --decode | jq -r ${1}
+					}
+
+					type=$(get_value '.type')
+					configs=$(get_value '.configurations')
+
+
+					for c in $(echo "${configs}" | jq -r '.[] | @base64'); do
+						get_value() {
+							echo ${c} | base64 --decode | jq -r ${1}
+						}
+
+						from_port=$(get_value '.from_port')
+						to_port=$(get_value '.to_port')
+						protocol=$(get_value '.protocol')
+						cidr_blocks=$(get_value '.cidr_blocks')
+						security_group+=$(cat <<_EOF
+
+	$type {
+		from_port	= $from_port
+		to_port		= $to_port
+		protocol	= "$protocol"
+		cidr_blocks	= ["$cidr_blocks"]
+	}
+_EOF
+	)
+					done
+				done
+			else
+				#TODO: remove this dirty hack
+				echo "Given security group will be used!"
+				is_array="false"
+				security_group=$(cat <<_EOF
+	ingress {
+		from_port       = 22
+		to_port         = 22
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+_EOF
+)
+			fi
+		else
+			echo "Custom security is not found! It will use default security group!"
+			security_group=$(cat <<_EOF
+	ingress {
+		from_port       = 22
+		to_port         = 22
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 80
+		to_port         = 80
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 443
+		to_port         = 443
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 51000
+		to_port         = 51100
+		protocol        = "tcp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	ingress {
+		from_port       = 51000
+		to_port         = 51100
+		protocol        = "udp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+	egress {
+		from_port       = 0
+		to_port         = 65535
+		protocol        = "udp"
+		cidr_blocks     = ["0.0.0.0/0"]
+	}
+_EOF
+)
+		fi
 
 	    aws_tf_content=$(cat <<_EOF
 variable "aws_region_$index" {
@@ -61,8 +166,8 @@ variable "aws_region_$index" {
 }
 
 variable "android_version_$index" {
-    type    = "string"
-    default = "$android_version"
+	type    = "string"
+	default = "$android_version"
 }
 
 variable "instance_type_$index" {
@@ -77,39 +182,7 @@ provider "aws" {
 
 resource "aws_security_group" "geny_sg_$index" {
 	provider = "aws.provider_$index"
-	ingress {
-		from_port       = 22
-		to_port         = 22
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-		description     = "SSH access"
-	}
-	ingress {
-		from_port       = 80
-		to_port         = 80
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-		description     = "HTTP access"
-	}
-	ingress {
-		from_port       = 443
-		to_port         = 443
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-		description     = "HTTPS access"
-	}
-	ingress {
-		from_port       = 51000
-		to_port         = 51100
-		protocol        = "tcp"
-		cidr_blocks     = ["0.0.0.0/0"]
-	}
-	ingress {
-		from_port       = 51000
-		to_port         = 51100
-		protocol        = "udp"
-		cidr_blocks     = ["0.0.0.0/0"]
-	}
+	$security_group
 }
 
 data "aws_ami" "geny_aws_$index" {
@@ -131,9 +204,9 @@ resource "aws_key_pair" "geny_key_$index" {
 
 resource "aws_instance" "geny_aws_$index" {
 	provider      = "aws.provider_$index"
-	ami           = "\${data.aws_ami.geny_aws_$index.id}"
+	ami="\${data.aws_ami.geny_aws_$index.id}"
 	instance_type = "\${var.instance_type_$index}"
-	vpc_security_group_ids = ["\${aws_security_group.geny_sg_$index.name}"]
+	vpc_security_group_ids=["\${aws_security_group.geny_sg_$index.name}"]
 	key_name      = "\${aws_key_pair.geny_key_$index.key_name}"
 	tags {
 		Name = "DockerAndroid-\${data.aws_ami.geny_aws_$index.id}"
@@ -157,6 +230,21 @@ output "public_dns_$index" {
 _EOF
 )
 		echo "$aws_tf_content" > /root/aws_tf_$index.tf
+
+		if [[ $ami != null ]]; then
+			echo "Using given AMI!"
+			sed -i "s/.*ami=.*/        ami=\"$ami\"/g" /root/aws_tf_$index.tf
+		else
+			echo "Custom AMI is not found. It will use the latest AMI!"
+		fi
+
+		if [[ $sg != null ]] && [[ $is_array == "false" ]]; then
+			echo "Using given security group: $sg"
+			sed -i "s/.*vpc_security_group_ids=.*/        vpc_security_group_ids=[\"$sg\"]/g" /root/aws_tf_$index.tf
+		fi
+
+		echo "---------------------------------------------------------"
+
 	    ((index++))
 	    ((port++))
 	done
@@ -200,6 +288,31 @@ function run_appium() {
   	$CMD
 }
 
+function ga(){
+	if [ "$GA" = true ]; then
+		echo "Collecting data for improving the project"
+		description="PROCESSOR: ${SYS_IMG}; VERSION: ${ANDROID_VERSION}; DEVICE: ${DEVICE}; APPIUM: ${APPIUM}; SELENIUM: ${CONNECT_TO_GRID}; MOBILE_TEST: ${MOBILE_WEB_TEST}"
+		random_user=$(cat /proc/version 2>&1 | sed -e 's/ /_/g' | sed -e 's/[()]//g' | sed -e 's/@.*_gcc_version/_gcc/g' | sed -e 's/__/_/g' | sed -e 's/Linux_version_//g' | sed -e 's/generic_build/genb/g')
+		random_user="${APP_RELEASE_VERSION}_${random_user}"
+		payload=(
+			--data v=${GA_API_VERSION}
+			--data aip=1
+			--data tid="${GA_TRACKING_ID}"
+			--data cid="${random_user}"
+			--data t="event"
+			--data ec="${APP_TYPE}"
+			--data ea="${random_user}"
+			--data el="${description}"
+			--data an="docker-android"
+			--data av="${APP_RELEASE_VERSION}"
+		)
+		curl ${GA_ENDPOINT} "${payload[@]}" --silent 
+	else
+		echo "Nothing to do"
+	fi
+}
+
+ga
 if [ "$REAL_DEVICE" = true ]; then
 	echo "Using real device"
 	run_appium

travis.sh

@@ -13,6 +13,9 @@ else
     elif [ ! -z "$REAL_DEVICE" ]; then
       echo "[SUPPORT FOR REAL DEVICE: BUILD DOCKER IMAGES AND PUSH THOSE TO DOCKER HUB ]"
       bash release_real.sh all $TRAVIS_TAG
+    elif [ ! -z "$GENYMOTION" ]; then
+      echo "[SUPPORT FOR GENYMOTION: BUILD DOCKER IMAGES AND PUSH THOSE TO DOCKER HUB ]"
+      bash release_genymotion.sh all $TRAVIS_TAG
     fi
     echo "Log out of docker hub"
     docker logout