Data collection and dataset preparation for Finetuning

[zalandAI]

Description:
We have gathered the latest documents related to KP Government policies, rules, and regulations. The next step is to prepare this dataset for fine-tuning the model.

Tasks:
Organise the Collected Data:

Verify all files are up-to-date and relevant.

Dataset Structuring:

Format Data for Fine-Tuning:

Convert data into a suitable format for the target model (e.g., JSON).

Acceptance Criteria:

• Data is organised and cleaned.
• The dataset is split into appropriate subsets.
• The data is formatted and ready for fine-tuning.