Fix GlueCrawlerOperature failure when using tags

[IAL32]

Closes #27556.
Previous implementation assumes that the current crawler data always has the keys in the wanted configuration, but this is not true, causing a KeyError when the key is not present in the current crawler configuration.

This PR fixes this by using the more stable dict.get method, providing a default value of None to allow proper comparison.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[IAL32]

Sorry for the force-push spam, I was sorting out the GPG signature thingy.

[Taragolis]

@IAL32 I also would be nice if we have some tests for this scenario

airflow/tests/providers/amazon/aws/hooks/test_glue_crawler.py

Line 86 in 9247258

class TestGlueCrawlerHook(unittest.TestCase):

[IAL32]

@Taragolis I have added several tests, and isolated tag updating by adding a update_tags method.
This is because when calling glue_client.get_crawler the Tags is not present, even if tags are correctly set for the given Crawler.
Furthermore, the update_crawler does not accept the Tags parameter: https://docs.aws.amazon.com/glue/latest/webapi/API_UpdateCrawler.html thus, making it impossible to update the tags for crawlers using conventional methods.

In my solution, I call tag_resource (https://docs.aws.amazon.com/glue/latest/webapi/API_TagResource.html) when adding/replacing existing tags, and untag_resource (https://docs.aws.amazon.com/glue/latest/webapi/API_UntagResource.html) when deleting tags.

[IAL32]

Changing tags however requires to know the crawler's ARN, which is only possible to obtain via concatenating the current account ID, the region name and the crawler's name.
The GlueCrawlerOperator now also requires the region_name in order to properly construct the GlueCrawlerHook. This is not different that what is currently done for GlueJobOperator and GlueJobHook.

[Taragolis]

I like your idea just some suggestions and nitpicks

[Taragolis]

IMHO, better not to add one hook property to another if it possible or at least make it "private"
Looks like we only need to call it once in GlueCrawlerHook, so we could directly call in update_tags method

account_id = StsHook(aws_conn_id=self.aws_conn_id).get_account_number()

As long term (separate PR) it is a good idea to create account_id property in AwsBaseHook.

[IAL32]

@Taragolis I updated the code following your suggestions 😄 thanks!

[potiuk]

Fixes needed :(

[IAL32]

@potiuk resolved conflicts and test

[IAL32]

Fixed more tests: https://github.com/apache/airflow/actions/runs/3622139044/jobs/6111701994

[boring-cyborg]

Awesome work, congrats on your first merged pull request!