Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E testing #6

Merged
merged 17 commits into from
Jun 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ export const EngineOverview: ReactFC<> = () => {
</h2>
</EuiTitle>
</EuiPageContentHeader>
<EuiPageContentBody>
<EuiPageContentBody data-test-subj="appSearchEngines">
<EngineTable
data={engines}
pagination={{
Expand All @@ -133,7 +133,7 @@ export const EngineOverview: ReactFC<> = () => {
</h2>
</EuiTitle>
</EuiPageContentHeader>
<EuiPageContentBody>
<EuiPageContentBody data-test-subj="appSearchMetaEngines">
<EngineTable
data={metaEngines}
pagination={{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,11 @@ export const EngineTable: ReactFC<IEngineTableProps> = ({
name: i18n.translate('xpack.appSearch.enginesOverview.table.column.name', {
defaultMessage: 'Name',
}),
render: name => <EuiLink {...engineLinkProps(name)}>{name}</EuiLink>,
render: (name) => (
<EuiLink data-test-subj="engineNameLink" {...engineLinkProps(name)}>
{name}
</EuiLink>
),
width: '30%',
truncateText: true,
mobileOptions: {
Expand Down
41 changes: 41 additions & 0 deletions x-pack/test/functional_enterprise_search/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Enterprise Search Functional E2E Tests

## Running these tests

Follow the [Functional Test Runner instructions](https://www.elastic.co/guide/en/kibana/current/development-functional-tests.html#_running_functional_tests).

There are two suites available to run, a suite that requires a Kibana instance without an `enterpriseSearch.host`
configured, and one that does. The later also [requires a running Enterprise Search instance](#enterprise-search-requirement), and a Private API key
from that instance set in an Environment variable.

Ex.

```sh
# Run specs from the x-pack directory
cd x-pack

# Run tests that require enterpriseSearch.host variable
APP_SEARCH_API_KEY=[use private key from local App Search instance here] node scripts/functional_tests --config test/functional_enterprise_search/with_host_configured.config.ts

# Run tests that do not require enterpriseSearch.host variable
APP_SEARCH_API_KEY=[use private key from local App Search instance here] node scripts/functional_tests --config test/functional_enterprise_search/without_host_configured.config.ts
```

## Enterprise Search Requirement

These tests will not currently start an instance of App Search automatically. As such, they are not run as part of CI and are most useful for local regression testing.

The easiest way to start Enterprise Search for these tests is to check out the `ent-search` project
and use the following script.

```sh
cd script/stack_scripts
/start-with-license-and-expiration.sh platinum 500000
```

Requirements for Enterprise Search:

- Running on port 3002 against a separate Elasticsearch cluster.
- Elasticsearch must have a platinum or greater level license (or trial).
- Must have Standard or Native Auth configured with an `enterprise_search` user with password `changeme`.
- There should be NO existing Engines or Meta Engines.
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import expect from '@kbn/expect';
import { EsArchiver } from 'src/es_archiver';
import { AppSearchService, IEngine } from '../../../../services/app_search_service';
import { Browser } from '../../../../../../../test/functional/services/browser';
import { FtrProviderContext } from '../../../ftr_provider_context';

export default function enterpriseSearchSetupEnginesTests({
getService,
getPageObjects,
}: FtrProviderContext) {
const esArchiver = getService('esArchiver') as EsArchiver;
const browser = getService('browser') as Browser;
const retry = getService('retry');
const appSearch = getService('appSearch') as AppSearchService;

const PageObjects = getPageObjects(['appSearch', 'security']);

describe('Engines Overview', function () {
let engine1: IEngine;
let engine2: IEngine;
let metaEngine: IEngine;

before(async () => {
await esArchiver.load('empty_kibana');
engine1 = await appSearch.createEngine();
engine2 = await appSearch.createEngine();
metaEngine = await appSearch.createMetaEngine([engine1.name, engine2.name]);
});

after(async () => {
await esArchiver.unload('empty_kibana');
appSearch.destroyEngine(engine1.name);
appSearch.destroyEngine(engine2.name);
appSearch.destroyEngine(metaEngine.name);
});

describe('when an enterpriseSearch.host is configured', () => {
it('navigating to the enterprise_search plugin will redirect a user to the App Search Engines Overview page', async () => {
await PageObjects.security.forceLogout();
const { user, password } = appSearch.getEnterpriseSearchUser();
await PageObjects.security.login(user, password, {
expectSpaceSelector: false,
});

await PageObjects.appSearch.navigateToPage();
await retry.try(async function () {
const currentUrl = await browser.getCurrentUrl();
expect(currentUrl).to.contain('/app_search');
});
});

it('lists engines', async () => {
const engineLinks = await PageObjects.appSearch.getEngineLinks();
const engineLinksText = await Promise.all(engineLinks.map((l) => l.getVisibleText()));

expect(engineLinksText.includes(engine1.name)).to.equal(true);
expect(engineLinksText.includes(engine2.name)).to.equal(true);
});

it('lists meta engines', async () => {
const metaEngineLinks = await PageObjects.appSearch.getMetaEngineLinks();
const metaEngineLinksText = await Promise.all(
metaEngineLinks.map((l) => l.getVisibleText())
);
expect(metaEngineLinksText.includes(metaEngine.name)).to.equal(true);
});
});
});
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { FtrProviderContext } from '../../ftr_provider_context';

export default function ({ loadTestFile }: FtrProviderContext) {
describe('Enterprise Search', function () {
loadTestFile(require.resolve('./app_search/engines'));
});
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import expect from '@kbn/expect';
import { FtrProviderContext } from '../../../ftr_provider_context';

export default function enterpriseSearchSetupGuideTests({
getService,
getPageObjects,
}: FtrProviderContext) {
const esArchiver = getService('esArchiver');
const browser = getService('browser');
const retry = getService('retry');

const PageObjects = getPageObjects(['appSearch']);

describe('Setup Guide', function () {
before(async () => await esArchiver.load('empty_kibana'));
after(async () => {
await esArchiver.unload('empty_kibana');
});

describe('when no enterpriseSearch.host is configured', () => {
it('navigating to the enterprise_search plugin will redirect a user to the setup guide', async () => {
await PageObjects.appSearch.navigateToPage();
await retry.try(async function () {
const currentUrl = await browser.getCurrentUrl();
expect(currentUrl).to.contain('/app_search/setup_guide');
});
});
});
});
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { FtrProviderContext } from '../../ftr_provider_context';

export default function ({ loadTestFile }: FtrProviderContext) {
describe('Enterprise Search', function () {
loadTestFile(require.resolve('./app_search/setup_guide'));
});
}
20 changes: 20 additions & 0 deletions x-pack/test/functional_enterprise_search/base_config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { FtrConfigProviderContext } from '@kbn/test/types/ftr';
import { pageObjects } from './page_objects';
import { services } from './services';

export default async function ({ readConfigFile }: FtrConfigProviderContext) {
const xPackFunctionalConfig = await readConfigFile(require.resolve('../functional/config'));

return {
// default to the xpack functional config
...xPackFunctionalConfig.getAll(),
services,
pageObjects,
};
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { FtrProviderContext } from '../ftr_provider_context';
import { TestSubjects } from '../../../../../test/functional/services/test_subjects';

export function AppSearchPageProvider({ getService, getPageObjects }: FtrProviderContext) {
const PageObjects = getPageObjects(['common']);
const testSubjects = getService('testSubjects') as TestSubjects;

return {
async navigateToPage() {
return await PageObjects.common.navigateToApp('app_search');
},

async getEngineLinks() {
const engines = await testSubjects.find('appSearchEngines');
return await testSubjects.findAllDescendant('engineNameLink', engines);
},
Comment on lines +19 to +22
Copy link
Owner

@cee-chen cee-chen May 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I've kinda had this conversation before but for Cypress, but I get mildly concerned with (what I think is?) over-DRYing stuff out instead of just using the native APIs inline.

  1. It requires someone new to the testing framework API to jump between 2 different files to understand what the helper is actually doing
  2. If we continue to use a getSomeSelector() helper for every single new test we add, this file is going to quickly balloon in size and become increasingly difficult to parse/read

This isn't a change request but more a general philosophy that but I'd prefer to keep selector helpers in the file they're used in, and only start DRYing them out to a shared file once more than 2-3+ tests duplicate them.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point, thanks.


async getMetaEngineLinks() {
const metaEngines = await testSubjects.find('appSearchMetaEngines');
return await testSubjects.findAllDescendant('engineNameLink', metaEngines);
},
};
}
13 changes: 13 additions & 0 deletions x-pack/test/functional_enterprise_search/page_objects/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { pageObjects as basePageObjects } from '../../functional/page_objects';
import { AppSearchPageProvider } from './app_search';

export const pageObjects = {
...basePageObjects,
appSearch: AppSearchPageProvider,
};
Loading