Query

[leosimony]

I am opening this issue here as you said. Sorry that I opened wrongly here https://github.com/aker-gateway/aker-freeipa-playbook/issues/7

Is this project being used in large numbers?
I am in desperate need of a gateway server for our Infra.

I have configured the Json method Aker and its working as expected. But for 500+ servers and growing and dynamic user config Infra, it's not easy to modify JSON files.

I don't have any idea about FreeIPA. Read like bind packages and IPA server needs DNS configs and all. We already running local DNS with dnsmasq.
Will it be an issue if we choose exeternal DNS for freeIPA.
Hosts and users management can be done in gui?

[leosimony]

get your point and there should be a tool add/del/edit entries.

What are the features you want to see in such tool?
csv import maybe?

That being said, this issue is opened against the wrong repo - this is the repo fro the ansible playbook installer - may I ask you to open an issue here instead.

---

Thank you for the above reply anazmy.
CSV import will be good I guess.
We use to deploy/remove servers frequently and give access to different teams. It is difficult to keep track about what access provided to whom and for what server. Making it in excel sheet is another headache.

To be frank, I thought FreeIPA as a GUI tool for managing the users and hosts but later on, I found its different and will not suit our environment. FreeIPA client is not directly supported on Debian distributions whereas all our servers are based on Debian.

[anazmy]

I get your point about FreeIPA, it's a a whole ecosystem that you need to deploy.

[leosimony]

Yup, its a whole separate system and not easy to use it on the already implemented setup.
Anything can be done for this? It would be good if there is. Please let me know

[leosimony]

is there any possibility of having CSV import or any other method to add and remove entries ?

[anazmy]

Yes, that's what I've in mind. I'm currently on travel and will try working on this when am back in around a week.

[leosimony]

Hello Anazmy
Have you had time to work on this ?

[leosimony]

Anazmy,
I have been looking for your reply to this. Please let me know if you have any info.

[anazmy]

Apologies for the delay.
Unfortunately am completely occupied with no free time to continue working on this. I will return to it in a future time.

[leosimony]

Oops.. I was expecting this. No issues. Thank you

…

On Tue, Oct 29, 2019, 9:46 AM anazmy ***@***.***> wrote: Apologies for the delay. Unfortunately am completely occupied with no free time to continue working on this. I will return to it in a future time. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#94?email_source=notifications&email_token=ALKF2HOXKMY4225MNVZBLETQQ62KXA5CNFSM4IJDNRH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECPFWPA#issuecomment-547248956>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALKF2HKRK72VO3VWMSKCC5TQQ62KXANCNFSM4IJDNRHQ> .

[anazmy]

Thanks for understanding. Meanwhile if you have got a PR to add this feature I would test it and get it merged

[EoleDev]

Hi,
For my own use, I realized a Mariadb connector for Aker which is linked to a database format.
With this you would be able to populate the database with the tool you want.
Will it be a solution for you ?
What would be the information you would be able to set ?

[anazmy]

Hi @EoleDev do you have the mariadb code some available somewhere to look at?

[EoleDev]

Not currently because it is using some proprietary information.
I will try to do a cleanup soon, to expose a generic connector.

[EoleDev]

Hi @anazmy,
You may find the MariaDB connector code on a dedicated branch on my fork : https://github.com/EoleDev/Aker/tree/mariadb
If all seems good for you, I may open a pull request.

[leosimony]

@EoleDev you have tried with Mariadb IDP in your setup and got the server list connecting to gateway server?

I have pulled the MariaDB branch from your repo and tried setting up the MariaDB idp but it is not working for me. The error message says as "ERROR - MARIADB: Could not connect to database, error :" though I am able to connect to the database via cli.

I am not sure whether the branch you have done is ready to use. I was looking for some change to this IDP, so I tried it once I saw this. Sorry if I tried too early.

[EoleDev]

Hi @leosimony,
I did a test with the Mariadb IDP and successfully retrieved the server list on aker.
It seems the error is not with the IDP but with the connection to your database.
Could you paste your cli command ?
and your aker ini file, with sensible data evicted of course.
The current branch which have 4 commits should be working.

[EoleDev]

Hi @leosimony
I did another check, because it seems you had no error displayed.
So now there is two more commits (6 in total).
One is a correction for the error not displayed in logs, and the other a little fix with
an undefined function (but you should not had a problem with it).
Now you may have in your logs the error encountered by Mariadb IDP when connecting
to the server.

[anazmy]

Hey thx @EoleDev for the awesome work!
I did test the mariadb branch and it looks good.
Can you please add some logging (debug/info) to help with the mariadb connector debugging?

BTW, how big is the environment that you tested that on?
I'm wondering about the JOIN performance in big environments.

[EoleDev]

Hi @anazmy,
Thank you for the test!
I will add today some debug/info.

I didn't test this connector in a production environment.
But concerning the JOIN performance, we are using MariaDB solution
on a lot of website, application and other stuff. We use a lot of JOIN request
on database and tables of more than 100000+ (I think it's 1 000 000+) entry
without performance issue. So I don't think it will be an issue!

[EoleDev]

@anazmy
I took a look at the Json IDP.
I added the same debug information, when they were relevant.
Feel free to ask for more if needed.

[leosimony]

Hi @leosimony
I did another check, because it seems you had no error displayed.
So now there is two more commits (6 in total).
One is a correction for the error not displayed in logs, and the other a little fix with
an undefined function (but you should not had a problem with it).
Now you may have in your logs the error encountered by Mariadb IDP when connecting
to the server.

Yes, I got it working @EoleDev.
Thank you for the awesome work.
@anazmy Thank you for the awesome tool you have created.

To both,
Is there any plan to do the phase1 and phase2 implementations as mentioned in the readme. Most importantly, prevention of executing rm -rf commands etc.

[EoleDev]

Considering the phase defined by @anazmy, I am not planning to help on all the things just because It would be quite complex, and I don't quite see who will use it.
Here some informations :

Phase 0
    Integration with an identity provider (FreeIPA) -> I think it's done
    Extendable Modular structure, plugin your own module -> If it is IDP, it is done
    Integration with config management tools -> I don't know if it is done, and it would need a list 
       of management tools supported
    Parsable audit logs (json, shipped to Elasticsearch) -> It is done, I am using it
    Highly available setup -> It would not be quite a problem, if the IDP are supporting it
    Session playback -> It is done, but could some enhancement to support elasticsearch

Phase 1
    Admin WebUI -> It would be quite a problem, because it would depend on the IDPs
    Live session monitoring -> It seems complex, be maybe in future I would work on it for personal use
    Cloud support (AWS,OpenStack etc..) or On-premises deployments -> Could we not already deploy it on cloud ?
    Command filtering (Prevent destructive commands like rm -rf) -> It would be possible
    Encrypt sessions logs stored on disk. -> It would be possible

Phase 2
    Support for graphical protocols (RDP, VNC, X11) monitoring -> I don't really know if someone would use it. And it would be a huge rework.
    User productivity dashboard -> What would be the information which should be displayed ? In fact someone could use elasticsearch and create its own dashboard for it.

For information, I implemented the support for the sftp protocol in Aker. It is not so user friendly for the connection, and if I remember well, I need to use a patched ssh client (due to the fact that they have an issue, they have not corrected on production and I rely on it). I will need to do a cleanup of the code, and to document its use for my own use. When it will be done, I may propose a PR.

Maybe the different phase could be modified, and if there is some other thing which would be important, I may help implement it.

[EoleDev]

I forgot to mention, I also have a patch to allow the use of multiple IDP.
It allow to have some servers on one IDP and some on another. But the user see the full list on connection.

[leosimony]

@EoleDev
That is really great news. I am really happy to see that @anazmy got some help at the end for developing his great work.

About the features listed, I would like to have the below whenever its possible. That will be a great addition to this aker gateway and will be one of the main reasons for one to consider using this setup.

Command filtering (Prevent destructive commands like rm -rf) -> It would be possible

[leosimony]

@EoleDev
I have tried the Mariadb IDP and my observations.

Tables:
hostgroups - id and hostgroup name
hosts - id, name, hostname
hosts_hostgroups - hostid and hostgroupid
hosts_usergroups - hostid and usergroupid
usergroups - id, usergroup name
users - id, username, keypath
users_usergroups - userid, usergroupid

If I have 200 hosts, I can add it to the hosts table using a csv export and that is a 1 minute job.

Hurdle:
We have multiple departments,
Infra - Should have access to all the hosts
Dev - should have access to particular hosts
Devops- should have access to particular hosts

Managing these in DB tables in an environment like us seems to be hard when there are many servers.
-->Servers will be deployed and deleted often
--> users will be resigning and adding often

May be, the work flow in our environment does not suit the Aker gateway Working method. I am just updating this if incase someone know a way to manage this and not in a way of complaining the application. Thank you

[EoleDev]

@leosimony
I don't really understand the problem.

You will delete and/or add many servers quite often. It is not a problem with mariadb. You may do it.
You will delete and/or add users quite often. It is not a problem too.

Could you explain what you are trying to achieve and was is blocking you ?

I am currently using Aker with a pool of 400+ servers managed in a mariadb server. And I have no issue. We deploy at least 1 server per week, and there may be some servers deleted per week.

[leosimony]

@EoleDev
For ex:

Tables:
hostgroups
1 infra
2 devops
3 dev

hosts
1 server1 server1.com
2 server2 server2.com
3 server3 server3.com

hosts_hostgroups(mapping 3 hosts to Infra and Devops Hostgroup)
1 1
2 1
3 1
1 2
2 2
3 2

hosts_usergroups(mapping 3 hosts to Infra and Devops usergroups)
1 1
2 1
3 1
1 2
2 2
3 2

usergroups:
1 infra
2 devops
3 dev

users:
1 user1
2 user2
3 users3

users_usersgroups
1 1
2 2
1 3

Say If I have 400 hosts and,
All hosts should be accessed by Infra
150 hosts have to be accessed only by Dev and Devops
100 hosts has to be accessed by Devops and Infra
100 hosts has to be accessed by Infra and Dev

-I have to do these mappings in the tables by identifying the host id, userid, hostgroup id. Doing these from time to time looks difficult to me(may be only to me because I am lazy ☹️ )

[EoleDev]

You could just develop a little UI to manage the database and do this for you!

[leosimony]

you have replied, just like that :-)

[EoleDev]

Sorry, I don't understand your answer :D

[EoleDev]

@leosimony
If I develop an UI to manage the MariaDB IDP, would it be of interest for you ?
Which would you prefer :

• a cli command
• a web UI
  It would be quite simple. Just to allow managing users/hosts/groups without managing theirs ids.

[leosimony]

@EoleDev Of course and thank you. I would prefer a simple Web UI to manage users/hosts/groups.

[EoleDev]

I will take the time to do it.
It will not be a beautiful UI as I am not an UX designer :D
It will be functional.

[leosimony]

I will take the time to do it.
It will not be a beautiful UI as I am not an UX designer :D

This is all everyone need 👍

It will be functional.

[anazmy]

@EoleDev
I have tried the Mariadb IDP and my observations.

Tables:
hostgroups - id and hostgroup name
hosts - id, name, hostname
hosts_hostgroups - hostid and hostgroupid
hosts_usergroups - hostid and usergroupid
usergroups - id, usergroup name
users - id, username, keypath
users_usergroups - userid, usergroupid

If I have 200 hosts, I can add it to the hosts table using a csv export and that is a 1 minute job.

Hurdle:
We have multiple departments,
Infra - Should have access to all the hosts
Dev - should have access to particular hosts
Devops- should have access to particular hosts

Managing these in DB tables in an environment like us seems to be hard when there are many servers.
-->Servers will be deployed and deleted often
--> users will be resigning and adding often

May be, the work flow in our environment does not suit the Aker gateway Working method. I am just updating this if incase someone know a way to manage this and not in a way of complaining the application. Thank you

Would this workflow make sense?

• Add a server
• Add this server to one or more hostgroups
• Add a user (or use an existing user)
• Add this user to one or more usergroups
• Assign usergroups permissions to certain hostgroups

[anazmy]

I will take the time to do it.
It will not be a beautiful UI as I am not an UX designer :D
It will be functional.

Thank you so much @EoleDev for all the effort ur putting here.
Perhaps you can can join the project?

[leosimony]

@EoleDev
I have tried the Mariadb IDP and my observations.
Tables:
hostgroups - id and hostgroup name
hosts - id, name, hostname
hosts_hostgroups - hostid and hostgroupid
hosts_usergroups - hostid and usergroupid
usergroups - id, usergroup name
users - id, username, keypath
users_usergroups - userid, usergroupid
If I have 200 hosts, I can add it to the hosts table using a csv export and that is a 1 minute job.
Hurdle:
We have multiple departments,
Infra - Should have access to all the hosts
Dev - should have access to particular hosts
Devops- should have access to particular hosts
Managing these in DB tables in an environment like us seems to be hard when there are many servers.
-->Servers will be deployed and deleted often
--> users will be resigning and adding often
May be, the work flow in our environment does not suit the Aker gateway Working method. I am just updating this if incase someone know a way to manage this and not in a way of complaining the application. Thank you

Would this workflow make sense?

• Add a server
• Add this server to one or more hostgroups
• Add a user (or use an existing user)
• Add this user to one or more usergroups
• Assign usergroups permissions to certain hostgroups

Yes @anazmy. I hope @EoleDev will be following the same work flow.

[EoleDev]

@anazmy I would be glad to join the project. But I have not so much time so I can't promise to do all the patch quickly.
The patches I released quickly were already made for me ;)

@leosimony and @anazmy
I was not thinking of the same workflow. I thought about the Hostgroups just as a mean to group the servers in the Aker UI. The usergroups are here for the rights definition. For me, a user is a member of some usergroups, and this fact allow him to have access to the hosts in the respective usergroups. These hosts, in the aker UI will be grouped by hostgroup.

[leosimony]

@EoleDev this also sounds okay. As logs as it serves the purpose then its fine.

[EoleDev]

@leosimony I began the development, I will try to finish it as soon as possible.

[EoleDev]

@leosimony @anazmy
The first version for the Mariadb IDP UI is done!!
you may find it here : https://github.com/EoleDev/aker-ui
Hope it will suit your needs.

[leosimony]

This is really great news. I will check it out and update you. Thank you

…

On Sat, Feb 29, 2020, 5:03 PM EoleDev ***@***.***> wrote: @leosimony <https://github.com/leosimony> @anazmy <https://github.com/anazmy> The first version for the Mariadb IDP UI is done!! you may find it here : https://github.com/EoleDev/aker-ui Hope it will suit your needs. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#94?email_source=notifications&email_token=ALKF2HJSTDLEMZZIOFOWTOTRFDZBRA5CNFSM4IJDNRH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENLXY7A#issuecomment-592936060>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALKF2HMDQM7Q2ZBCRHRSAJDRFDZBRANCNFSM4IJDNRHQ> .

[leosimony]

@anazmy You have any idea to Merge this Aker-UI to be a part of the Aker project. I think that will help users looking for an SSH gateway project like this.

[EoleDev]

@leosimony
As it is a different project which rely on a specific IDP... IMHO we should not merge them.
But we could put some information about it in the README of aker.
It would allow people to know it exists.

[leosimony]

@EoleDev Oh okay. yes, if it is mentioned on the Readme, people will know.
I have updated the ansible-playbook I was using before to install Elasticsearch 6.x version, Aker gateway with MariaDB idp and Aker-UI as well.

[EoleDev]

@leosimony Good!
In a near future, I could add a panel to the current UI to replay session from elasticsearch in the browser.
I currently have a POC but I will need some time because I am quite busy right now.

[anazmy]

Thx @leosimony for the wonderful contribution!

Pls allow me sometime to go through your additions

[EoleDev]

Hi @anazmy,
As I was looking on the old issues, I was asking myself if you were asking me
some access on the source of the UI and IDP I developped.
If True, which access would you need ?