feat(SemanticLayerSchema): making column type optional and avoid dumping and parsing null values in the SemanticLayerSchema

[scaliseraoul]

• All code checks passed.

---

Important

Make column type optional in SemanticLayerSchema and avoid dumping/parsing null values, with related documentation and code updates.

• Behavior:
  • type in Column class in semantic_layer_schema.py is now optional.
  • params in Transformation class in semantic_layer_schema.py is now optional.
  • to_dict() method in SemanticLayerSchema now excludes None values.
  • to_yaml() method in SemanticLayerSchema uses to_dict() to exclude None values.
• Documentation:
  • Updated docs/v3/semantic-layer.mdx to reflect changes in column definition from dict to list.
  • Updated docs/v3/contributing.mdx to change make tests to make test_all.
• Code Improvements:
  • Changed _get_abs_dataset_path() in loader.py to return a string path.
  • Updated _create_yml_template() in base.py to handle optional columns.
  • Updated test_save_creates_correct_schema() in test_dataframe.py to reflect schema changes.

^{This description was created by for f889c86. It will automatically update as commits are pushed.}

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to f889c86 in 1 minute and 38 seconds

More details

• Looked at 254 lines of code in 6 files
• Skipped 0 files when reviewing.
• Skipped posting 5 drafted comments based on config settings.

1. docs/v3/contributing.mdx:67

• Draft comment:
  Ensure that the change from make tests to make test_all is reflected in the actual Makefile or test script to maintain consistency.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The change in the test command from make tests to make test_all in the documentation should be reflected in the actual Makefile or test script. This ensures consistency between the documentation and the actual commands used.

2. pandasai/dataframe/base.py:179

• Draft comment:
  Check if columns_dict is not None before mapping it to Column objects to avoid potential TypeError.
• Reason this comment was not posted:
  Comment looked like it was already resolved.

3. pandasai/data_loader/semantic_layer_schema.py:38

• Draft comment:
  Ensure that params is checked for None before accessing its elements to avoid potential AttributeError.
• Reason this comment was not posted:
  Comment did not seem useful.

4. pandasai/data_loader/semantic_layer_schema.py:27

• Draft comment:
  Check if type is not None before validating it to avoid potential TypeError.
• Reason this comment was not posted:
  Marked as duplicate.

5. pandasai/data_loader/semantic_layer_schema.py:30

• Draft comment:
  Consider rephrasing the error message for unsupported column type for clarity:

                f"Unsupported column type: {type}. Supported types are: {', '.join(VALID_COLUMN_TYPES)}."

• Reason this comment was not posted:
  Comment looked like it was already resolved.

Workflow ID: wflow_8URjNb3RqkoKGaja

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.