Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Wallet] E2E test improvements #2542

Merged
merged 13 commits into from
Jan 27, 2020
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ export PATH=$ANDROID_HOME/emulator:$ANDROID_HOME/tools:$PATH
Run the emulator with:

```bash
emulator -avd Nexus_5X_API_28
emulator -avd Nexus_5X_API_28_x86
```

#### Optional: Install Genymotion Emulator Manager
Expand Down
2 changes: 1 addition & 1 deletion packages/mobile/.env
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ENVIRONMENT=local
DEFAULT_TESTNET=alfajoresstaging
DEFAULT_TESTNET=integration
jmrossy marked this conversation as resolved.
Show resolved Hide resolved
# If ZERO_SYNC_ENABLED_INITIALLY, local geth will not run initially.
# If toggled on, it will use DEFAULT_SYNC_MODE. See src/geth/consts.ts for more info
ZERO_SYNC_ENABLED_INITIALLY=false
Expand Down
3 changes: 2 additions & 1 deletion packages/mobile/.env.test
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ DEFAULT_TESTNET=integration
ZERO_SYNC_ENABLED_INITIALLY=true
DEFAULT_SYNC_MODE=5
DEV_SETTINGS_ACTIVE_INITIALLY=true
# Disable firebase b.c. google-services.json files are missing in CI
FIREBASE_ENABLED=false
SECRETS_KEY=debug
SHOW_TESTNET_BANNER=false
SHOW_GET_INVITE_LINK=true
SHOW_GET_INVITE_LINK=true
24 changes: 2 additions & 22 deletions packages/mobile/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,13 +88,6 @@ renders when no state has changed. Reducing renders can be done via pure
components in react or overloading the should component update method
[example here][rn optimize example].

### Connecting to networks

By default, we have the `alfajores` network set up. If you have other testnets
that you want to use with the app, update `.env.ENV-NAME` and `packages/mobile/.env.ENV-NAME` with
the new network name and settings, then rebuild the app. Note that this will assume the testnets
have a corresponding `/blockchain-api` and `/notification-service` set up.

### Running Wallet app in zero sync (Data Saver) mode

By default, the mobile wallet app runs geth in ultralight sync mode where all the epoch headers are fetched. The default sync mode is defined in [packages/mobile/.env](https://github.com/celo-org/celo-monorepo/blob/master/packages/mobile/.env#L4) file.
Expand Down Expand Up @@ -130,21 +123,9 @@ See [`src/identity/verification.test.ts`] for an example.
### E2E testing

We use [Detox][detox] for E2E testing. In order to run the tests locally, you
must have the proper emulator set up. Emulator installation instructions are in
the [setup instructions][setup#emulator].

Please set `123456` as the pin code in the emulator, since the e2e tests rely on
that.

Next, the VM snapshot settings should be modified:

1. Close all apps and lock the emulator (go to lock screen).
2. Power off the emulator
3. Power it back on and go to emulator settings (... button) -> Snapshots -> Settings
4. Set Auto-Save to No
must have the proper emulator set up. Follow the instrutions in [e2e/README.md][e2e readme].

For information on how to run and extend the e2e tests, refer to the
[e2e readme][e2e readme].
Once setup is done, you can run the tests with `yarn test:e2e:android`

## Building APKs / Bundles

Expand Down Expand Up @@ -264,7 +245,6 @@ $ adb kill-server && adb start-server
[rn profiler]: https://reactjs.org/blog/2018/09/10/introducing-the-react-profiler.html
[rn running on device]: https://facebook.github.io/react-native/docs/running-on-device
[setup]: ../../SETUP.md
[setup#emulator]: ../../SETUP.md#optional-install-an-android-emulator
[react-native-testing-library]: https://github.com/callstack/react-native-testing-library
[rntl-docs]: https://callstack.github.io/react-native-testing-library/
[jest]: https://jestjs.io/docs/en/snapshot-testing
Expand Down
6 changes: 2 additions & 4 deletions packages/mobile/android/app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -252,10 +252,8 @@ android {
packagingOptions {
exclude 'META-INF/-no-jdk.kotlin_module'
exclude 'META-INF/androidx.exifinterface_exifinterface.version'
pickFirst 'lib/x86/libc++_shared.so'
pickFirst 'lib/arm64-v8a/libc++_shared.so'
pickFirst 'lib/x86_64/libc++_shared.so'
pickFirst 'lib/armeabi-v7a/libc++_shared.so'
pickFirst '**/libjsc.so'
pickFirst '**/libc++_shared.so'
}


Expand Down
3 changes: 3 additions & 0 deletions packages/mobile/android/app/proguard-rules.pro
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@

# Add any project specific keep options here:

# Detox
-keep class com.facebook.react.ReactInstanceManager { *; }

# RN Firebase
-keep class io.invertase.firebase.** { *; }
-dontwarn io.invertase.firebase.**
Expand Down
2 changes: 1 addition & 1 deletion packages/mobile/android/app/src/debug/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@

<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW"/>

<application android:usesCleartextTraffic="true" tools:targetApi="28" tools:ignore="GoogleAppIndexingWarning" />
<application android:usesCleartextTraffic="true" tools:ignore="GoogleAppIndexingWarning" android:networkSecurityConfig="@xml/network_security_config" />
</manifest>
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<!-- This allows unencrypted communication to these domains, which the bundler needs when running on an emulator -->
<domain-config cleartextTrafficPermitted="true">
<domain includeSubdomains="true">localhost</domain>
<domain includeSubdomains="true">10.0.1.1</domain>
<domain includeSubdomains="true">10.0.2.2</domain>
<domain includeSubdomains="true">10.0.3.2</domain>
</domain-config>
</network-security-config>
68 changes: 35 additions & 33 deletions packages/mobile/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,46 +2,21 @@

[![e2e test status](https://storage.googleapis.com/celo-e2e-data/e2e-banner.svg)](https://console.cloud.google.com/storage/browser/celo-e2e-data?project=celo-testnet)

These are the End-to-End (e2e) tests for the mobile (payments) application. They run an emulator and simulate a user clicking through the mobile app.
These are the End-to-End (e2e) tests for the wallet mobile app. They run an emulator and simulate a user clicking through the app.

## The e2e banner
## Setting up the VM

In the readme files (in the root, mobile, and this one), there are banners for the e2e tests. The test status is saved in a [google cloud storage bucket](https://console.cloud.google.com/storage/browser/celo-e2e-data?project=celo-testnet).
There is also a log file for the last test run.
First install the emulator as described in the [SETUP readme](../../../SETUP.md#optional-install-an-android-emulator).

Too see all the versions of the log file:
By default, the e2e scripts will assume the VM name of `Nexus_5X_API_28_x86` recommended in the instructions but you can rename the VM as you like.

```bash
gsutil ls -al gs://celo-e2e-data/last_run_log
```
Next, to improve reliability of the tests, configure the VM as described in the [Detox best practices doc](https://github.com/wix/Detox/blob/master/docs/Introduction.AndroidEmulatorsBestPractices.md).

Too display a specific version of the log file:
## Running the tests

```bash
gsutil cat gs://celo-e2e-data/last_run_log#<version_number>
gsutil cat gs://celo-e2e-data/last_run_log #specify no version number to get the latest
```
Simply run `yarn test:e2e:android` or `yarn test:e2e:ios`

If you need to have a more detailed look, there is a collection of log files and even screenshots for the failing tests saved in `detailed_logs.tar.gz`. Download with:

```bash
gsutil cp gs://celo-e2e-data/detailed_logs.tar.gz .
tar -xvf detailed_logs.tar.gz
```

These files are uploaded by by the [a script](../scripts/ci-e2e.sh), that is executed regularly. Don't use this script to run the tests locally.

## Running the tests locally

First install the emulator as described in the [mobile readme](../README.md).

```bash
yarn test:build-e2e #To build the java parts
yarn test:run-e2e #packages the react native parts of the app, launches the emulator and runs the tests
```

After making small changes to the app or adding new tests you can rerun the tests without rebuilding first.
When the tests are running, they will automatically try to enter `123456` as the pin code when needed. You will have to set this pin code in the settings of the emulator.
The run_e2e.sh script will take care of configuring and building the app for you.

## Adding a test

Expand Down Expand Up @@ -107,6 +82,33 @@ The e2e tests should use as few mocks as possible, since they are supposed to be

The mocks are only used, when the environment variable `CELO_TEST_CONFIG` is set too 'e2e'. This variable will be read in `mobile/rn-cli.config.js` and will modify what the metro bundler will include in the bundle. If you're mocking a module from node_nodules, put the mock in `e2e/mocks/`. Use the file extension `.e2e.ts` or `.e2e.js`.

## The e2e banner

In the readme files (in the root, mobile, and this one), there are banners for the e2e tests. The test status is saved in a [google cloud storage bucket](https://console.cloud.google.com/storage/browser/celo-e2e-data?project=celo-testnet).
There is also a log file for the last test run.

Too see all the versions of the log file:

```bash
gsutil ls -al gs://celo-e2e-data/last_run_log
```

Too display a specific version of the log file:

```bash
gsutil cat gs://celo-e2e-data/last_run_log#<version_number>
gsutil cat gs://celo-e2e-data/last_run_log #specify no version number to get the latest
```

If you need to have a more detailed look, there is a collection of log files and even screenshots for the failing tests saved in `detailed_logs.tar.gz`. Download with:

```bash
gsutil cp gs://celo-e2e-data/detailed_logs.tar.gz .
tar -xvf detailed_logs.tar.gz
```

These files are uploaded by by the [a script](../scripts/ci-e2e.sh), that is executed regularly. Don't use this script to run the tests locally.

## Troubleshooting

If tests are failing, and you don't why:
Expand Down
22 changes: 0 additions & 22 deletions packages/mobile/e2e/Transfer.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -93,28 +93,6 @@ describe('Transfer Works', () => {
await sleep(10000)
})

it.skip('NUX->ImportContacts', async () => {
await device.launchApp({ permissions: { contacts: 'YES' } })

await waitFor(element(by.id('ImportContactsPermissionTitle')))
.toBeVisible()
.withTimeout(1000)

await waitFor(element(by.id('importContactsEnable')))
.toBeVisible()
.withTimeout(1000)

await waitFor(element(by.id('importContactsSkip')))
.toBeVisible()
.withTimeout(1000)

if (ENABLE_CONTACT_IMPORT) {
await element(by.id('importContactsEnable')).tap()
} else {
await element(by.id('importContactsSkip')).tap()
}
})

it.skip('NUX->VerifyEducation', async () => {
await waitFor(element(by.id('VerifyEducationHeader')))
.toBeVisible()
Expand Down
13 changes: 11 additions & 2 deletions packages/mobile/e2e/init.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,23 @@ const detox = require('detox')
const config = require('../package.json').detox
const adapter = require('detox/runners/jest/adapter')
const specReporter = require('detox/runners/jest/specReporter')
const assignReporter = require('detox/runners/jest/assignReporter')

jest.setTimeout(120000)
jasmine.getEnv().addReporter(adapter)
// This takes care of generating status logs on a per-spec basis. By default, jest only reports at file-level.
// This is strictly optional.
jasmine.getEnv().addReporter(specReporter)

// This will post which device has assigned to run a suite, which can be useful in a multiple-worker tests run.
// This is strictly optional.
jasmine.getEnv().addReporter(assignReporter)

// Increase default jest timeout
jest.setTimeout(120000)

beforeAll(async () => {
await detox.init(config)
})
}, 300000)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition! We should add a comment as to why this is needed.
From what I saw, jest swallows errors or timeouts here and happily continues with executing the test suite though this crucial init failed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure, I'm just aligning this file with the example on Detox's repo


beforeEach(async () => {
await adapter.beforeEach()
Expand Down
15 changes: 6 additions & 9 deletions packages/mobile/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,8 @@
"test:ci": "yarn test --coverage --runInBand",
"test:watch": "yarn test --watch --maxWorkers=4",
"test:verbose": "export TZ=UTC && jest --verbose",
"test:build-e2e": "bash ./scripts/build_e2e.sh",
"test:run-e2e": "bash ./scripts/run_e2e.sh",
"test:dry-run-e2e": "yarn dev:emulator && cd android && yarn test:build-e2e && yarn test:run-e2e",
"test:detox": "CELO_TEST_CONFIG=e2e detox test -c android.emu.debug -a e2e/tmp/ --take-screenshots=failing --record-logs=failing --detectOpenHandles -l verbose",
"test:unlock": "./scripts/unlock.sh",
"test:e2e:android": "./scripts/run_e2e.sh -p android",
"test:e2e:ios": "./scripts/run_e2e.sh -p ios",
"test:verify-locales": "./scripts/verify_locales.sh",
"pre-deploy": "./scripts/pre-deploy.sh",
"deploy:update-disclaimer": "yarn licenses generate-disclaimer > android/app/src/main/assets/custom/LicenseDisclaimer.txt"
Expand All @@ -43,7 +40,7 @@
"@celo/client": "0.0.247",
"@celo/contractkit": "0.2.10-dev",
"@celo/react-components": "1.0.0",
"@celo/react-native-fast-crypto": "^1.8.2",
"react-native-fast-crypto": "git+https://github.com/celo-org/react-native-fast-crypto#3946f9",
"@celo/react-native-sms-retriever": "git+https://github.com/celo-org/react-native-sms-retriever#b88e502",
"@celo/utils": "0.1.6-dev",
"@celo/walletkit": "^0.0.15",
Expand Down Expand Up @@ -81,7 +78,7 @@
"react-i18next": "^11.2.7",
"react-native": "^0.61.2",
"react-native-android-open-settings": "^1.3.0",
"react-native-bip39": "git://github.com/celo-org/react-native-bip39#8d00738",
"react-native-bip39": "git://github.com/celo-org/react-native-bip39#b31a8ff",
"react-native-camera": "^3.7.1",
"react-native-clock-sync": "^1.0.0",
"react-native-config": "https://github.com/luggit/react-native-config#89a602b",
Expand Down Expand Up @@ -163,7 +160,7 @@
"@types/utf8": "^2.1.6",
"@types/web3": "^1.0.18",
"babel-core": "7.0.0-bridge.0",
"detox": "^14.5.0",
"detox": "^15.0.0",
"enzyme": "^3.9.0",
"enzyme-adapter-react-16": "^1.11.2",
"enzyme-to-json": "^3.3.5",
Expand All @@ -189,7 +186,7 @@
"configurations": {
"android.emu.debug": {
"binaryPath": "android/app/build/outputs/apk/debug/app-debug.apk",
"build": "cd android && ./gradlew assembleDebug assembleAndroidTest -DtestBuildType=debug && cd ..",
"build": "cd android && ENVFILE=.env.test ./gradlew assembleDebug assembleAndroidTest -DtestBuildType=debug && cd ..",
"type": "android.emulator",
"name": "Nexus_5X_API_28_x86",
"__comment": "Nexus_5X_API_28_x86 is currently harcoded but it should be the result from $ANDROID_SDK_ROOT/emulator/emulator -list-avds | grep 'x86' | head -n 1`"
Expand Down
15 changes: 0 additions & 15 deletions packages/mobile/scripts/build_e2e.sh

This file was deleted.

Loading