[2nd iteration] Soft deprecation for ".wazuh" index #1771

jesusgn90 · 2019-09-03T11:46:35Z

Hi team,

After #1465 and after discussing internally how we should manage this task, here are the next steps for this task.

Client side (front-end)

If there are no API entries, show a guide. @jesusgn90
- Explain how to add an API entry.
- Add an option for testing the connection in the same guide.
If there are API entries, but we can't reach the API, show a guide. @jesusgn90
- This guide shows how to check the API service, how to configure an API entry.
- This guide also shows the existing API entries plus a button for checking the connection of all of them.
Remove the form in Settings @jesusgn90
- Also change the API entries table (no actions except "test connection" and "set as default")

Server side (back-end)

Default structure for wazuh-hosts.yml:

# Wazuh API entries
wazuh.hosts:
#  - <id>:
#     url: <url>
#     port: <port>
#     user: <user>
#     password: <password>

Real example:

# Wazuh API entries
wazuh.hosts:
  - production:
     url: https://10.0.0.2
     port: 55000
     user: admin
     password: kj178hkjasdf2
  - staging:
     url: http://localhost
     port: 55000
     user: foo
     password: bar

In the above example, both production and staging are API entries. Each one would have an entry in the wazuh-registry.json with the extensions and the cluster information.

The text was updated successfully, but these errors were encountered:

jesusgn90 · 2019-09-05T15:54:42Z

Hi team, I've been working on the front-end side of this task. The working branch is https://github.com/wazuh/wazuh-kibana-app/tree/issue-1771-1.

Started two new components for "How to configure the Wazuh API" and "Troubleshooting Wazuh API not reachable", not finished yet, but here is a sneak-peek:

On the other hand, I'm also working on changing the "Settings" controller, some functions are no longer needed, other ones are wrong now and must be adapted for this task.

adri9valle · 2019-09-06T13:27:30Z

Update 09.06.19

The following points related to this issue are solved in this branch: migrate-index-yml.

If the index .wazuh exists, copy its content to wazuh-hosts.yml
- Decode base64 passwords from API entries.
- Ensure all API entries were copied properly before removing the index.
- Remove the index once done.
- Do not duplicate entries.

Working on: Every API entry found in wazuh-hosts.yml has a pair in wazuh-registry.json (old wazuh-version.json) with the cluster status, the manager name, and the related extensions.

jesusgn90 · 2019-09-06T15:46:13Z

After the view changes on the client-side, we've refactored the controller dcee4c1 to remove unused methods, and to adapt the controller for latest changes.

adri9valle · 2019-09-09T12:54:23Z

Update 09.09.19

The wazuh-registry.json is updated with the cluster information and extensions of the migrated host once the host is migrated from the index to the registry file.

Every API entry found in wazuh-hosts.yml has a pair in wazuh-registry.json (old wazuh-version.json) with the cluster status, the manager name, and the related extensions.
(In progress) Adapt our server endpoints for updating API properties, the front-end should be able to use the same endpoints as before, not changing its logic drastically.
- Added an endpoint to get the hosts and the related information about them:

[   
    {  
      "yqYyFm0BvJqWYtQxtyi9":{  
         "url":"http://localhost",
         "port":55000,
         "user":"foo",
         "password":"******",
         "cluster_info":{  
            "manager":"master",
            "node":"manager_01",
            "cluster":"wazuh",
            "status":"enabled"
         },
         "extensions":{  
            "audit":true,
            "pci":true,
            "gdpr":true,
            "hipaa":true,
            "nist":true,
            "oscap":false,
            "ciscat":false,
            "aws":false,
            "virustotal":false,
            "osquery":false,
            "docker":false
         }
      }
   },
   {  
      "y6YyFm0BvJqWYtQxuChI":{  
         "url":"http://localhost",
         "port":55000,
         "user":"foo",
         "password":"******",
         "cluster_info":{  
            "manager":"master",
            "node":"manager_01",
            "cluster":"wazuh",
            "status":"enabled"
         },
         "extensions":{  
            "audit":true,
            "pci":true,
            "gdpr":true,
            "hipaa":true,
            "nist":true,
            "oscap":false,
            "ciscat":false,
            "aws":false,
            "virustotal":false,
            "osquery":false,
            "docker":false
         }
      }
   }
]

adri9valle · 2019-09-10T14:31:25Z

Updated 10.09.19

The API entries in the Settings > API section are loaded from the wazuh-hosts.yml.
When checking connection the registry is updated instead of update the index.

adri9valle · 2019-09-11T15:14:58Z

Updated 11.09.19

Working on: Adapt our server endpoints for updating API properties, the front-end should be able to use the same endpoints as before, not changing its logic drastically.

Some endpoints have been changed:

Updated the selected API.
Check manager connection.
Update cluster information.

adri9valle · 2019-09-12T15:05:10Z

Update 12.09.19

Working on: Adapt our server endpoints for updating API properties, the front-end should be able to use the same endpoints as before, not changing its logic drastically.

Adapt wazuh-api.js controller for using wazuh-hosts.yml
Adapt settings-wizard
Adapt api-count

adri9valle · 2019-09-13T07:12:48Z

Update 13.09.19

Adapt our server endpoints for updating API properties, the front-end should be able to use the same endpoints as before, not changing its logic drastically.
- Removed wazuh-api-elastic backend endpoints and controllers.
- Cleaned the initialize in order to remove unused and deprecated functions.
Remove orphan registry entries when a host is deleted:

Now every time that the API entries are requested in Settings > API the registry host entries are checked in order to compare with the wazuh-hosts entries to remove the orphan registers.
Removed orphan code.
Check the manager connection before setting as default.
Checks for new APIs and the connection of them.

adri9valle · 2019-09-16T15:14:39Z

Update 16.09.19

Modified the api-table component.
Some fixes in the settings-wizard.
Fix when joining the hosts with the registry entries.
Add API component first poc:

Cannot reach any API:

Some API can be reached others cannot:

All API entries are reached:

adri9valle · 2019-09-17T15:05:01Z

Update 17.09.19

Improve checks for new APIs
Test if API is down in wazuh-api
Check down APIs improved(now when test the API connectivity you can know if is not reachable or is down).
Api is down component finish(can be changed):

adri9valle · 2019-09-18T15:00:05Z

Update 18.09.19

Merged with 3.11-7.3
Prevent create wazuh-registry.json without hosts
Improve check manager, now it sets the API status(online, offline, unknown)
Check APIs status when getting them
Change the way to display the helpers components
Improve api-is-down component
Adapt settings-wizard

Current status:

adri9valle · 2019-09-19T15:54:31Z

Update 19.09.19

Improved several functionalities, also, add the refresh of the entries:

adri9valle · 2019-09-20T13:42:34Z

Update 20.09.19

Fixed api-count.js. The object structure for the API changed and it was trying to get undefined fields, this was provoking that the first time that navigates to another section different to Settings the App redirects to Settings again.

Removed wazuh-hosts.yml now we're using the wazuh.yml to store the hosts instead.

adri9valle · 2019-09-30T14:58:04Z

Update 30.09.19

The api-is-down component only was checking the initial API entries, now on each check gets again the API entries in the wazuh.yml in order to check there is some new API entry or some entry was removed.
Settings-wizard and api-count only tried to set as default the first entry, if this entry cannot be selected as default the APP was redirecting to Settings, now both try to select as default the first entry that can be recheabled.
Fixed several typos and flicks.
The api-table component lists the API entries when refreshing if the yml is bad formatted the error was not handled.

adri9valle · 2019-10-01T13:28:54Z

Update 01.10.19

The error handler has been improved, until now when getting some error in the YML parser the component only was showing that an error occurred but not the reason why now this error is shown:

Also, the monitoring script has been adapted.

adri9valle · 2019-10-02T10:26:52Z

Update 02.10.19

Unsigned certifications are now is supported.

adri9valle · 2019-10-04T10:54:13Z

Update 04.10.19

Fixed:

When there's not a default API selected in the cookies the health-check when starting the APP thrown an error.
When any API entry could be selected the APP was allowing the navigation, now always redirect to settings in order to manage the API entries and solve the problem.
Warning when the select API changes.
When migrating from the index, if the hosts: key exists but there're not hosts the key was duplicated provoking a parser error.

NOTE:

If there's some API entry as default in the cookies when accessing the APP if the credentials have changed or the entry has been removed an error is shown in the health-check, this is normal behavior because the stored API cannot be connected, also, is the current behavior.

adri9valle · 2019-10-07T10:35:04Z

Update 07.10.19

Improved error handler when stored API cannot be reached.

adri9valle · 2019-10-08T06:53:30Z

Update 08.10.19

Added the ID to the tables.
Added the capability to search in the table.
Added the capability to sort in the table.
Added spinner while checking API connection.

adri9valle · 2019-10-09T09:15:26Z

Update 09.10.19

Added spinner also in api-table instead the message refreshing API entries.
Show a tooltip with the reason why the API cannot be reachable.
Add click action in the ? to copy the error in the clipboard.

adri9valle · 2019-10-09T10:21:23Z

Solved: #1811

jesusgn90 added type/enhancement Enhancement issue back-end Thins related to the server side priority/medium labels Sep 3, 2019

jesusgn90 assigned adri9valle and jesusgn90 Sep 3, 2019

jesusgn90 added this to the 36th week sprint milestone Sep 3, 2019

adri9valle modified the milestones: 36th week sprint, Sprint - 100 Sep 11, 2019

adri9valle mentioned this issue Sep 20, 2019

Migrate API entries from .wazuh index to wazuh.yml #1811

Merged

juankaromo modified the milestones: Sprint - 100, Sprint - 101 Sep 30, 2019

adri9valle closed this as completed Oct 1, 2019

adri9valle reopened this Oct 1, 2019

adri9valle closed this as completed Oct 9, 2019

jsanchez91 mentioned this issue Oct 9, 2019

Backports 6.8.3 #1826

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[2nd iteration] Soft deprecation for ".wazuh" index #1771

[2nd iteration] Soft deprecation for ".wazuh" index #1771

jesusgn90 commented Sep 3, 2019 •

edited by adri9valle

Loading

jesusgn90 commented Sep 5, 2019

adri9valle commented Sep 6, 2019 •

edited

Loading

jesusgn90 commented Sep 6, 2019

adri9valle commented Sep 9, 2019 •

edited

Loading

adri9valle commented Sep 10, 2019

adri9valle commented Sep 11, 2019

adri9valle commented Sep 12, 2019

adri9valle commented Sep 13, 2019 •

edited

Loading

adri9valle commented Sep 16, 2019

adri9valle commented Sep 17, 2019

adri9valle commented Sep 18, 2019

adri9valle commented Sep 19, 2019

adri9valle commented Sep 20, 2019 •

edited

Loading

adri9valle commented Sep 30, 2019 •

edited

Loading

adri9valle commented Oct 1, 2019 •

edited

Loading

adri9valle commented Oct 2, 2019 •

edited

Loading

adri9valle commented Oct 4, 2019 •

edited

Loading

adri9valle commented Oct 7, 2019

adri9valle commented Oct 8, 2019 •

edited

Loading

adri9valle commented Oct 9, 2019 •

edited

Loading

adri9valle commented Oct 9, 2019

[2nd iteration] Soft deprecation for ".wazuh" index #1771

[2nd iteration] Soft deprecation for ".wazuh" index #1771

Comments

jesusgn90 commented Sep 3, 2019 • edited by adri9valle Loading

Client side (front-end)

Server side (back-end)

jesusgn90 commented Sep 5, 2019

adri9valle commented Sep 6, 2019 • edited Loading

jesusgn90 commented Sep 6, 2019

adri9valle commented Sep 9, 2019 • edited Loading

adri9valle commented Sep 10, 2019

adri9valle commented Sep 11, 2019

adri9valle commented Sep 12, 2019

adri9valle commented Sep 13, 2019 • edited Loading

adri9valle commented Sep 16, 2019

adri9valle commented Sep 17, 2019

adri9valle commented Sep 18, 2019

adri9valle commented Sep 19, 2019

adri9valle commented Sep 20, 2019 • edited Loading

adri9valle commented Sep 30, 2019 • edited Loading

adri9valle commented Oct 1, 2019 • edited Loading

adri9valle commented Oct 2, 2019 • edited Loading

adri9valle commented Oct 4, 2019 • edited Loading

adri9valle commented Oct 7, 2019

adri9valle commented Oct 8, 2019 • edited Loading

adri9valle commented Oct 9, 2019 • edited Loading

adri9valle commented Oct 9, 2019

jesusgn90 commented Sep 3, 2019 •

edited by adri9valle

Loading

adri9valle commented Sep 6, 2019 •

edited

Loading

adri9valle commented Sep 9, 2019 •

edited

Loading

adri9valle commented Sep 13, 2019 •

edited

Loading

adri9valle commented Sep 20, 2019 •

edited

Loading

adri9valle commented Sep 30, 2019 •

edited

Loading

adri9valle commented Oct 1, 2019 •

edited

Loading

adri9valle commented Oct 2, 2019 •

edited

Loading

adri9valle commented Oct 4, 2019 •

edited

Loading

adri9valle commented Oct 8, 2019 •

edited

Loading

adri9valle commented Oct 9, 2019 •

edited

Loading