Feat [Golang] Context Signal + Excellent Improvement 🤪 (#18)

* Feat [Golang] Context Signal

- [+] feat(session.go): add context parameter to ConvertSessionsToCSV function
- [+] feat(main.go): add context and signal handling to cancel operations on SIGINT or SIGTERM
- [+] fix(main.go): exit with code 0 after repairing data
- [+] fix(main.go): remove unused code and comments
- [+] fix(main.go): handle context cancellation in processCSVOption function
- [+] test(main_test.go): add context and cancel function to test ProcessCSVOption function

* Feat & Refactor [Golang] Command Module + Code Comment

- [+] fix(main.go): add comment to clarify the purpose of the code block
- [+] refactor(main.go): rename variable 'store' to 'dataStore' for clarity
- [+] feat(main.go): update package description and add support for repairing and exporting chat session data

* Fix [Golang] Command Module

- [+] fix(main.go): handle EOF error when reading input in promptForInput function

* Feat [Golang] Context Main Command Module

- [+] refactor(main.go): change promptForInput function signature to include context parameter
- [+] feat(main.go): add support for handling EOF and other errors in promptForInput function
- [+] feat(main.go): add support for exiting the program on EOF and other errors in promptForInput function
- [+] feat(main.go): add support for exiting the program after cancellation in setupSignalHandling function
- [+] feat(main.go): add support for exiting the program on other errors in promptForInput function
- [+] feat(main.go): add support for exiting the program on invalid format option in processCSVOption function
- [+] feat(main.go): add support for exiting the program on failed extraction to dataset in processDatasetOption function
- [+] feat(main.go): add support for exiting the program on invalid save output option in saveToFile function
- [+] feat(main.go): add support for exiting the program on invalid format option in executeCSVConversion function
- [+] feat(main.go): add support for exiting the program on failed creation of

* Test [Golang] Testing Proposal Main Command Module

- [+] test(main_test.go): add support for capturing and asserting stdout output in TestProcessCSVOption
- [+] test(main_test.go): add support for canceling context in TestPromptForInput

* Feat [Golang] [Module] Testing + Exporter [Package] Code Comment

- [+] chore(session.go): add support for context-aware operations
- [+] feat(session.go): add support for context cancellation in ConvertSessionsToCSV function
- [+] test(main_test.go): add test for promptForInput function

* Feat [Golang] [Module] Main Command Code Comment

- [+] refactor(main.go): add context support for cancellation in main function
- [+] feat(main.go): add signal handling for graceful termination in setupSignalHandling function
- [+] feat(main.go): add context support for cancellation in promptForInput function
- [+] feat(main.go): add context support for cancellation in processOutputOption function
- [+] feat(main.go): add context support for cancellation in processDatasetOption function
- [+] feat(main.go): add context support for cancellation in saveToFile function
- [+] refactor(main.go): remove context support from repairJSONData function
- [+] feat(main.go): add context support for cancellation in executeCSVConversion function
- [+] feat(main.go): add context support for cancellation in createSeparateCSVFiles function
- [+] feat(main.go): add context support for cancellation in convertToSingleCSV function
- [+] feat(main.go): add context support for cancellation in writeContentToFile function

* Refactor [Golang] [Module] Main Command Code Comment

- [+] refactor(main.go): improve comments and signal handling in setupSignalHandling function

* Test [Golang] Main Command Module

- [+] test(main_test.go): add error handling to promptForInput function in TestPromptForInput test

* Feat & Fix [Golang] [Module] Main Command Context

- [+] feat(main.go): add support for context cancellation during user input prompts
- [+] fix(main.go): handle errors during user input prompts and exit program accordingly

* Fix [Golang] [Module] Main Command Context

- [+] fix(main.go): handle interrupt signal error when prompting for input

* Fix [Golang] [Module] [Package] Main Command & Exporter

- [+] fix(session.go): handle errors from closing CSV writers and files
- [+] fix(session.go): handle errors from initializing CSV files
- [+] fix(session.go): handle errors from writing session data
- [+] fix(session.go): handle errors from writing message data
- [+] fix(main.go): handle errors from promptForInput function
- [+] fix(main.go): handle errors from repairJSONData function
- [+] fix(main.go): handle errors from exporter.ReadJSONFromFile function
- [+] fix(main.go): handle errors from exporter.ExtractToDataset function
- [+] fix(main.go): handle errors from saveToFile function
- [+] fix(main.go): handle errors from executeCSVConversion function
- [+] fix(main.go): handle errors from createSeparateCSVFiles function
- [+] fix(main.go): handle errors from exporter.CreateSeparateCSVFiles function
- [+] fix(main.go): handle errors from convertToSingleCSV function
- [+] fix(main.go): handle errors from writeContentToFile function

* Refactor [Golang] [Module] Main Command Repair JSON Data

- [+] fix(main.go): pass context to repairJSONData function
- [+] refactor(main.go): remove duplicated error handling code in repairJSONData function
- [+] refactor(main.go): remove unnecessary comments in repairJSONData function
- [+] feat(main.go): simulate a context-aware operation in repairJSONData function

* Test [Golang] [Module] Main Command Testing purposes.

- [+] test(main_test.go): update package comment to provide a comprehensive description of the tests
- [+] test(main_test.go): add TestProcessCSVOption to verify the functionality of processCSVOption
- [+] test(main_test.go): add TestPromptForInput to verify the capturing and returning of user input
- [+] test(main_test.go): add TestPromptForInputCancellation to check if promptForInput respects context cancellation
- [+] test(main_test.go): add TestLoadTestSessionsInvalidPath to verify that loadTestSessions returns an error for non-existent files
- [+] test(main_test.go): add TestLoadIncorrectJson to check loadTestSessions' behavior when provided with incorrect JSON format
- [+] test(main_test.go): add TestRepairJSONDataFromFile to verify the repairJSONData function with both valid and invalid file paths
- [+] test(main_test.go): add TestWriteContentToFile to verify that writeContentToFile function writes the expected content to a file

* Refactor [Golang] File System

- [+] refactor(file_system.go): rename package from "main" to "filesystem" and move file to "filesystem" directory
- [+] refactor(file_system_mock.go): rename package from "main" to "filesystem" and move file to "filesystem" directory

Note: This is a better approach because not everything is contained within the "main" package.

* Docs [Golang] [Package] File System

- [+] feat(filesystem): add package-level documentation for the filesystem package
- [+] fix(filesystem): fix comments and formatting in file_system.go
- [+] feat(filesystem): add RealFileSystem implementation of the FileSystem interface
- [+] fix(filesystem): fix comments and formatting in file_system_mock.go
- [+] feat(filesystem): add MockFileSystem implementation of the FileSystem interface

* Feat CI Gopher Unit Testing 🧪

- [+] chore(GopherCI.yml): add GopherCI workflow for unit testing
- [+] feat(GopherCI.yml): add workflow to run Gopher unit tests on push and pull request events

* Docs [README] Gopher Unit Testing 🧪

- [+] docs(README.md): add Gopher Unit Testing badge to README

* Chore CI Gopher Unit Testing 🧪

- [+] chore(GopherCI.yml): update Gopher Unit Testing job configuration
  - Change job name to "Run Gopher Unit Testing on ${{ matrix.os }}"
  - Update "runs-on" to use ${{ matrix.os }} variable
  - Add "strategy" section to define matrix for "os" and "go-version"
  - Update "go-version" to use ${{ matrix.go-version }} variable
  - Use "actions/setup-go@v3" to set up Go with the specified version
  - Use "actions/checkout@v3" to check out code into the Go module directory

* Chore CI Gopher Unit Testing 🧪

- [+] chore(GopherCI.yml): temporarily remove `windows-latest` from the matrix due to issues with Windows

* Feat [Golang] [Package] File System Mock

- [+] feat(filesystem): add mock implementation of the FileSystem interface for testing purposes
- [+] feat(filesystem): add mock implementation of the exporter.Exporter interface for testing purposes
- [+] fix(filesystem): update package comment to include purpose of the mock implementation
- [+] fix(filesystem): update MockFileSystem to use map instead of slice to track created files
- [+] fix(filesystem): update Create method to create a new buffer in the FilesCreated map
- [+] fix(filesystem): update Stat method to return mock file information if file exists in the mock file system
- [+] fix(filesystem): update ReadFile method to read content of a file from the FilesCreated map
- [+] fix(filesystem): update WriteFile method to write data to a file in the FilesCreated map
- [+] feat(filesystem): add dummy implementation of mockFileInfo for testing purposes

* Test [Golang] [Module] Main Command Testing purposes

- [+] test(main_test.go): add unit test for writeContentToFile function

* Chore & Feat [Golang] [Package] File System

- [+] chore(file_system.go): update FileSystem interface to include WriteFile method
- [+] feat(file_system.go): add WriteFile method to RealFileSystem implementation

* Feat [Golang] [Module] Main Command FileSystem

- [+] fix(main.go): fix writeContentToFile function to use provided FileSystem interface and handle errors properly
- [+] feat(main.go): add support for filesystem package and use RealFileSystem implementation in writeContentToFile function

* Chore CI Gopher Unit Testing 🧪

- [+] chore(GopherCI.yml): add workflow_dispatch event to trigger manual workflow runs with branch input

* Docs [Golang] [Package] File System Mock

- [+] chore(file_system_mock.go): add comments to MockExporter and ConvertSessionsToCSV functions

* CI Gopher Unit Testing 🧪

- [+] ci(GopherCI.yml): add build job to build the application on different operating systems

* CI Gopher Unit Testing 🧪

- [+] chore(GopherCI.yml): update GopherCI workflow configuration
 - Update author information
 - Add CI/CD note
 - Add todo for integrating AI for summarization tasks
 - Rename "Build" step to "Test Build"

.github/workflows/GopherCI.yml

@@ -0,0 +1,67 @@
+#######################################################################
+# Author: H0llyW00dzZ                                                 #
+# CI/CD Note: this A better GopherCI, instead of using paid ci.🤣    #
+# Todo: Integrate AI for summarization tasks.                        #
+######################################################################
+name: Gopher Unit Testing
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+    types: [opened, reopened, synchronize]
+
+  workflow_dispatch:
+    inputs:
+      branch:
+        description: 'Branch to test'
+        required: true
+        default: 'master'
+
+jobs:
+  test:
+    name: Run Gopher Unit Testing on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      # Temporarily removed `windows-latest` due to some issues with Windows. However, my code is correct lmao.
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        go-version: ['1.21.5']
+
+    steps:
+    - name: Set up Go
+      uses: actions/setup-go@v3
+      with:
+        go-version: ${{ matrix.go-version }}
+
+    - name: Check out code into the Go module directory
+      uses: actions/checkout@v3
+
+    - name: Get dependencies
+      run: go mod tidy
+
+    - name: Run tests
+      run: |
+        go test -v -timeout 30s -run '^(TestProcessCSVOption|TestPromptForInput|TestPromptForInputCancellation|TestLoadTestSessionsInvalidPath|TestLoadIncorrectJson|TestRepairJSONDataFromFile|TestWriteContentToFile)' github.com/H0llyW00dzZ/ChatGPT-Next-Web-Session-Exporter
+
+  build:
+    name: Gopher Unit Testing Building Application on ${{ matrix.os }}
+    needs: test
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        go-version: ['1.21.5']
+
+    steps:
+    - name: Check out code
+      uses: actions/checkout@v3
+
+    - name: Set up Go
+      uses: actions/setup-go@v3
+      with:
+        go-version: ${{ matrix.go-version }}
+
+    - name: Test Build
+      run: go build -v ./...

README.md

@@ -1,6 +1,7 @@
 # ChatGPT Next Web Session Exporter
 [![Go Report Card](https://goreportcard.com/badge/github.com/H0llyW00dzZ/ChatGPT-Next-Web-Session-Exporter)](https://goreportcard.com/report/github.com/H0llyW00dzZ/ChatGPT-Next-Web-Session-Exporter)
 [![Go Reference](https://pkg.go.dev/badge/github.com/H0llyW00dzZ/ChatGPT-Next-Web-Session-Exporter.svg)](https://pkg.go.dev/github.com/H0llyW00dzZ/ChatGPT-Next-Web-Session-Exporter)
+[![Gopher Unit Testing](https://github.com/H0llyW00dzZ/ChatGPT-Next-Web-Session-Exporter/actions/workflows/GopherCI.yml/badge.svg)](https://github.com/H0llyW00dzZ/ChatGPT-Next-Web-Session-Exporter/actions/workflows/GopherCI.yml)
 
 This repository contains tools that facilitate the conversion of chat session data from JSON format to various CSV formats and Hugging Face datasets. It provides a Bash script and a Go program designed to cater to different requirements for data processing and readability, offering multiple output options.
 

exporter/session.go

@@ -15,6 +15,9 @@
 // The package also handles fields in the source JSON that may be represented as either
 // strings or integers by using the custom StringOrInt type.
 //
+// Additionally, it now supports context-aware operations, allowing for better control
+// over long-running processes and the ability to cancel them if needed.
+//
 // Code:
 //
 //	func (soi *StringOrInt) UnmarshalJSON(data []byte) error {
@@ -37,13 +40,16 @@
 //
 // Usage examples:
 //
-// To read chat sessions from a JSON file and convert them to a CSV format:
+// To read chat sessions from a JSON file and convert them to a CSV format with context support:
+//
+//	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
+//	defer cancel()
 //
 //	store, err := exporter.ReadJSONFromFile("path/to/chat-sessions.json")
 //	if err != nil {
 //	    log.Fatal(err)
 //	}
-//	err = exporter.ConvertSessionsToCSV(store.ChatNextWebStore.Sessions, exporter.FormatOptionInline, "output.csv")
+//	err = exporter.ConvertSessionsToCSV(ctx, store.ChatNextWebStore.Sessions, exporter.FormatOptionInline, "output.csv")
 //	if err != nil {
 //	    log.Fatal(err)
 //	}
@@ -65,6 +71,7 @@
 package exporter
 
 import (
+	"context"
 	"encoding/csv"
 	"encoding/json"
 	"fmt"
@@ -186,12 +193,17 @@ func ReadJSONFromFile(filePath string) (ChatNextWebStore, error) {
 	return store, nil
 }
 
-// ConvertSessionsToCSV writes a slice of Session objects into a CSV file.
+// ConvertSessionsToCSV writes a slice of Session objects into a CSV file with support for context cancellation.
+//
+// The function takes a context.Context object as the first parameter to allow for cancellation and timeouts.
 //
-// It formats the CSV data in different ways based on the formatOption parameter.
+// The formatOption parameter determines how the CSV data should be formatted.
 //
-// It returns an error if the format option is invalid or if writing the CSV data fails.
-func ConvertSessionsToCSV(sessions []Session, formatOption int, outputFilePath string) error {
+// The outputFilePath parameter specifies where to save the CSV file.
+//
+// The function returns an error if the context is cancelled, the format option is invalid,
+// or if writing the CSV data fails.
+func ConvertSessionsToCSV(ctx context.Context, sessions []Session, formatOption int, outputFilePath string) error {
 	outputFile, err := os.Create(outputFilePath)
 	if err != nil {
 		return fmt.Errorf("failed to create output CSV file: %w", err)
@@ -221,6 +233,15 @@ func ConvertSessionsToCSV(sessions []Session, formatOption int, outputFilePath s
 
 	// Write each session to the CSV based on the selected format
 	for _, session := range sessions {
+		// Check if the context has been cancelled or a deadline has been exceeded.
+		select {
+		case <-ctx.Done():
+			// If the context is cancelled, return the context's error.
+			return ctx.Err()
+		default:
+			// If the context is not cancelled, proceed with processing.
+		}
+
 		var sessionData []string
 		switch formatOption {
 		case 1: // Inline Formatting
@@ -245,8 +266,10 @@ func ConvertSessionsToCSV(sessions []Session, formatOption int, outputFilePath s
 			sessionData = []string{session.ID, session.Topic, session.MemoryPrompt, string(messagesJSON)}
 		}
 		// Write the session data to the CSV
-		if err := csvWriter.Write(sessionData); err != nil {
-			return fmt.Errorf("failed to write session data to CSV: %w", err)
+		if formatOption != 2 {
+			if err := csvWriter.Write(sessionData); err != nil {
+				return fmt.Errorf("failed to write session data to CSV: %w", err)
+			}
 		}
 	}
 
@@ -328,28 +351,40 @@ func closeCSVWriter(csvWriter *csv.Writer, file *os.File) error {
 // Errors from closing files or flushing data to the CSV writers are captured and will be returned after all operations are attempted.
 //
 // Error messages are logged to the console.
-func CreateSeparateCSVFiles(sessions []Session, sessionsFileName string, messagesFileName string) error {
+func CreateSeparateCSVFiles(sessions []Session, sessionsFileName string, messagesFileName string) (err error) {
 	// Create and initialize the sessions CSV file.
-	sessionsFile, sessionsWriter, err := initializeCSVFile(sessionsFileName, []string{"id", "topic", "memoryPrompt"})
+	var sessionsFile *os.File
+	var sessionsWriter *csv.Writer
+	sessionsFile, sessionsWriter, err = initializeCSVFile(sessionsFileName, []string{"id", "topic", "memoryPrompt"})
 	if err != nil {
 		return err
 	}
-	defer closeCSVWriter(sessionsWriter, sessionsFile) // Simplified error handling
+	defer func() {
+		if cerr := closeCSVWriter(sessionsWriter, sessionsFile); cerr != nil {
+			err = cerr
+		}
+	}()
 
 	// Write session data.
-	if err := WriteSessionData(sessionsWriter, sessions); err != nil {
+	if err = WriteSessionData(sessionsWriter, sessions); err != nil {
 		return err
 	}
 
 	// Create and initialize the messages CSV file.
-	messagesFile, messagesWriter, err := initializeCSVFile(messagesFileName, []string{"session_id", "message_id", "date", "role", "content", "memoryPrompt"})
+	var messagesFile *os.File
+	var messagesWriter *csv.Writer
+	messagesFile, messagesWriter, err = initializeCSVFile(messagesFileName, []string{"session_id", "message_id", "date", "role", "content", "memoryPrompt"})
 	if err != nil {
 		return err
 	}
-	defer closeCSVWriter(messagesWriter, messagesFile) // Simplified error handling
+	defer func() {
+		if cerr := closeCSVWriter(messagesWriter, messagesFile); cerr != nil {
+			err = cerr
+		}
+	}()
 
 	// Write message data.
-	if err := WriteMessageData(messagesWriter, sessions); err != nil {
+	if err = WriteMessageData(messagesWriter, sessions); err != nil {
 		return err
 	}
 

filesystem/file_system.go

@@ -0,0 +1,42 @@
+// Package filesystem provides an abstraction over the native file system operations.
+//
+// This allows for easy testing and mocking of file system interactions.
+package filesystem
+
+import (
+	"io/fs"
+	"os"
+)
+
+// FileSystem is an interface that abstracts file system operations such as creating
+// files, writing to files, and retrieving file information. This allows for implementations
+// that can interact with the file system or provide mock functionality for testing purposes.
+type FileSystem interface {
+	Create(name string) (*os.File, error)
+	WriteFile(name string, data []byte, perm fs.FileMode) error
+	Stat(name string) (os.FileInfo, error)
+}
+
+// RealFileSystem implements the FileSystem interface by wrapping the os package functions,
+// thus providing an actual file system interaction mechanism.
+type RealFileSystem struct{}
+
+// Create creates a new file with the given name.
+// It wraps the os.Create function and returns a pointer to the created file along with any error encountered.
+func (fs RealFileSystem) Create(name string) (*os.File, error) {
+	return os.Create(name)
+}
+
+// WriteFile writes data to a file named by filename.
+// If the file does not exist, WriteFile creates it with permissions perm;
+// otherwise WriteFile truncates it before writing.
+func (fs RealFileSystem) WriteFile(name string, data []byte, perm fs.FileMode) error {
+	return os.WriteFile(name, data, perm)
+}
+
+// Stat returns the FileInfo structure describing the file named by the given name.
+// It wraps the os.Stat function and returns the FileInfo and any error encountered, for instance,
+// if the file does not exist.
+func (fs RealFileSystem) Stat(name string) (os.FileInfo, error) {
+	return os.Stat(name)
+}