diff --git a/README.md b/README.md index 089fa77..72bd022 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,44 @@ A Go implementation of the Model Context Protocol (MCP), providing a framework for building MCP servers and clients. +### Installation + +To install the `go-go-mcp` command line tool with homebrew, run: + +```bash +brew tap go-go-golems/go-go-go +brew install go-go-golems/go-go-go/go-go-mcp +``` + +To install the `go-go-mcp` command using apt-get, run: + +```bash +echo "deb [trusted=yes] https://apt.fury.io/go-go-golems/ /" >> /etc/apt/sources.list.d/fury.list +apt-get update +apt-get install go-go-mcp +``` + +To install using `yum`, run: + +```bash +echo " +[fury] +name=Gemfury Private Repo +baseurl=https://yum.fury.io/go-go-golems/ +enabled=1 +gpgcheck=0 +" >> /etc/yum.repos.d/fury.repo +yum install go-go-mcp +``` + +To install using `go get`, run: + +```bash +go get -u github.com/go-go-golems/go-go-mcp/cmd/go-go-mcp +``` + +Finally, install by downloading the binaries straight from [github](https://github.com/go-go-golems/go-go-mcp/releases). + ## Overview This project implements the [Model Context Protocol](https://github.com/modelcontextprotocol/specification), which enables standardized communication between AI applications and language models. The implementation includes: @@ -154,10 +192,10 @@ Start the server with either stdio or SSE transport: ```bash # Start with stdio transport (default) -./go-go-mcp start --transport stdio +go-go-mcp start --transport stdio # Start with SSE transport -./go-go-mcp start --transport sse --port 3001 +go-go-mcp start --transport sse --port 3001 ``` #### Client Mode @@ -166,26 +204,26 @@ Use the client subcommand to interact with an MCP server: ```bash # List available prompts (uses default server: go-go-mcp start --transport stdio) -./go-go-mcp client prompts list +go-go-mcp client prompts list # List available tools -./go-go-mcp client tools list +go-go-mcp client tools list # Execute a prompt with arguments -./go-go-mcp client prompts execute hello --args '{"name":"World"}' +go-go-mcp client prompts execute hello --args '{"name":"World"}' # Call a tool with arguments -./go-go-mcp client tools call echo --args '{"message":"Hello, MCP!"}' +go-go-mcp client tools call echo --args '{"message":"Hello, MCP!"}' ``` You can customize the server command and arguments if needed: ```bash # Use a different server binary with custom arguments -./go-go-mcp client --command custom-server,start,--debug,--port,8001 prompts list +go-go-mcp client --command custom-server,start,--debug,--port,8001 prompts list # Use a server with a specific configuration -./go-go-mcp client -c go-go-mcp,start,--config,config.yaml prompts list +go-go-mcp client -c go-go-mcp,start,--config,config.yaml prompts list ``` #### Using SSE Transport @@ -194,18 +232,34 @@ For web-based applications, use the SSE transport: ```bash # Start the server with SSE transport -./go-go-mcp start --transport sse --port 3001 +go-go-mcp start --transport sse --port 3001 # In another terminal, connect using the client -./go-go-mcp client --transport sse --server http://localhost:3001 prompts list +go-go-mcp client --transport sse --server http://localhost:3001 prompts list ``` +#### Using go-go-mcp as a Bridge + +go-go-mcp can be used as a bridge to expose an SSE server as a stdio server. This is useful when you need to connect a stdio-based client to an SSE server: + +```bash +# Start an SSE server on port 3000 +go-go-mcp start --transport sse --port 3000 + +# In another terminal, start the bridge to expose the SSE server as stdio +go-go-mcp bridge --sse-url http://localhost:3000 --log-level debug + +# Now you can use stdio-based clients to connect to the bridge +``` + +This is particularly useful when integrating with tools that only support stdio communication but need to connect to a web-based MCP server. + ### Debug Mode Add the `--debug` flag to enable detailed logging: ```bash -./go-go-mcp client --debug prompts list +go-go-mcp client --debug prompts list ``` ### Version Information @@ -213,33 +267,123 @@ Add the `--debug` flag to enable detailed logging: Check the version: ```bash -./go-go-mcp version +go-go-mcp version ``` -### Project Structure +## Configuration -- `pkg/` - - `protocol/` - Core protocol types and interfaces - - `prompts/` - Prompt registry and handlers - - `resources/` - Resource registry and handlers - - `tools/` - Tool registry and handlers - - `server/` - Server implementation - - `doc/` - Documentation -- `cmd/` - - `mcp-server/` - MCP server and client implementation - - `cmds/` - Command implementations - - `client/` - Client subcommands +go-go-mcp can be configured using YAML configuration files that allow you to: +- Define multiple profiles for different environments +- Configure tool and prompt sources +- Set parameter defaults and overrides +- Control access through blacklists/whitelists +- Manage security through parameter filtering -### Dependencies +For detailed configuration documentation, use: +```bash +# View configuration file documentation +go-go-mcp help config-file + +# View example configuration +go-go-mcp help config-file --example +``` + +## Shell Commands + +go-go-mcp supports defining custom shell commands in YAML files, providing: +- Template-based command generation with Go templates and Sprig functions +- Rich parameter type system +- Environment variable management +- Working directory control +- Error handling and output capture -- `github.com/rs/zerolog` - Structured logging -- `github.com/spf13/cobra` - Command-line interface -- Standard Go libraries +### Example Commands + +The `examples/` directory contains various ready-to-use commands. You can view the schema for any command using `go-go-mcp schema `, which shows the full parameter documentation that is passed to the LLM: + +```bash +go-go-mcp schema examples/github/list-github-issues.yaml +``` -## Contributing +#### GitHub Integration +- [`examples/github/list-github-issues.yaml`](examples/github/list-github-issues.yaml) - List and filter GitHub issues +- [`examples/github/list-pull-requests.yaml`](examples/github/list-pull-requests.yaml) - List and filter pull requests -Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change. +#### Web Content Tools +- [`examples/shell-commands/fetch-url.yaml`](examples/shell-commands/fetch-url.yaml) - Fetch and process web content +- [`examples/html-extract/pubmed.yaml`](examples/html-extract/pubmed.yaml) - Search and extract data from PubMed -## License +#### Research and Documentation +- [`examples/shell-commands/diary-append.yaml`](examples/shell-commands/diary-append.yaml) - Maintain a timestamped diary +- [`examples/shell-commands/fetch-transcript.yaml`](examples/shell-commands/fetch-transcript.yaml) - Download YouTube video transcripts -[Insert your chosen license here] \ No newline at end of file +For any command, you can view its full schema and documentation using: +```bash +# View the full parameter schema and documentation +go-go-mcp schema examples/github/list-github-issues.yaml + +# View the command help +go-go-mcp run-command examples/github/list-github-issues.yaml --help +``` + +### Running Shell Commands Directly + +You can run shell commands directly using the `run-command` subcommand. This allows you to execute any shell command YAML file without loading it into a server first: + +```bash +# View command help and available flags +go-go-mcp run-command examples/github/list-github-issues.yaml --help + +# Run a command with specific parameters +go-go-mcp run-command examples/github/list-github-issues.yaml --author wesen + +# Run a Google Calendar command +go-go-mcp run-command examples/google/get-calendar.yaml --start today --end "next week" + +# Run a URL fetching command +go-go-mcp run-command examples/shell-commands/fetch-url.yaml --url https://example.com +``` + +This provides a simpler way to use shell commands as standalone command-line tools without setting up a server. + +### Generating Shell Commands with Pinocchio + +You can use [Pinocchio](https://github.com/go-go-golems/pinocchio) to generate shell commands for go-go-mcp. First, add your local go-go-mcp repository as a Pinocchio repository: + +```bash +pinocchio config repositories add $(pwd)/pinocchio +``` + +Then generate a new command using: + +```bash +pinocchio go-go-mcp create-command --description "A command description" +``` + +This will create a new shell command YAML file with the appropriate structure and configuration. + +For detailed shell commands documentation, use: +```bash +# View shell commands documentation +go-go-mcp help shell-commands + +# View example shell commands +go-go-mcp help shell-commands --example +``` + +## Help System + +go-go-mcp comes with comprehensive built-in documentation. To explore: + +```bash +# List all available help topics +go-go-mcp help --all + +# Get help on a specific topic +go-go-mcp help + +# Show examples for a topic +go-go-mcp help --example +``` + +### Project Structure diff --git a/changelog.md b/changelog.md index 672eb7e..2431864 100644 --- a/changelog.md +++ b/changelog.md @@ -859,4 +859,33 @@ Added notification handling support to the stdio transport: - Added notification handler to StdioTransport struct - Implemented SetNotificationHandler method - Added notification detection and handling in Send method -- Improved response handling to properly handle interleaved notifications \ No newline at end of file +- Improved response handling to properly handle interleaved notifications + +## Documentation: Added Pinocchio Integration Instructions + +Added documentation about using Pinocchio to generate shell commands for go-go-mcp. + +- Added section about adding go-go-mcp as a Pinocchio repository +- Added instructions for using the create-command template + +## Documentation: Added Run Command Usage Instructions + +Added documentation about using the run-command subcommand to execute shell commands directly as standalone command-line tools. + +- Added section about using run-command with shell command YAML files +- Added examples for different types of commands (GitHub, Google Calendar, URL fetching) + +## Documentation: Added Example Commands Overview + +Added a comprehensive overview of example commands available in the examples directory. + +- Added categorized listing of example commands (GitHub, Google Calendar, Web Content, etc.) +- Added brief descriptions for each command +- Improved discoverability of available tools + +## Documentation: Added Schema Command and Improved Example Links + +Enhanced the example commands documentation: +- Added information about using the schema command to view parameter documentation +- Converted example command paths to clickable markdown links +- Added examples of viewing command documentation using both schema and help \ No newline at end of file diff --git a/doc/shell-commands.md b/doc/shell-commands.md index a09b0bb..fd86e47 100644 --- a/doc/shell-commands.md +++ b/doc/shell-commands.md @@ -3,141 +3,74 @@ Shell commands allow you to define executable commands and scripts in YAML files, with support for templated arguments, environment variables, and flexible execution options. ## Table of Contents -1. [Basic Structure](#basic-structure) -2. [Command Types](#command-types) - - [Command Lists](#command-lists) - - [Shell Scripts](#shell-scripts) -3. [Parameters](#parameters) - - [Flags](#flags) - - [Arguments](#arguments) -4. [Environment Variables](#environment-variables) -5. [Working Directory](#working-directory) -6. [Output Handling](#output-handling) -7. [Examples](#examples) - - [Simple Command](#simple-command) - - [Complex Script](#complex-script) - - [Docker Operations](#docker-operations) - - [Git Operations](#git-operations) - - [Database Operations](#database-operations) - -## Basic Structure - -A shell command YAML file has the following basic structure: +1. [Command Structure](#command-structure) +2. [Parameter Types](#parameter-types) +3. [Templating System](#templating-system) +4. [Examples](#examples) +5. [Best Practices](#best-practices) +6. [Troubleshooting](#troubleshooting) + +## Command Structure + +A shell command YAML file has the following structure: ```yaml -name: command-name -short: Short description -long: | +# Metadata (Required) +name: command-name # Command name (use lowercase and underscores) +short: Short description # One-line description +long: | # Optional: Detailed multi-line description Detailed description that can span multiple lines -flags: - - name: flag_name - type: string - help: Flag description - required: true -command: - - executable + +# Parameter Definition +flags: # Optional: Command parameters + - name: flag_name # Required: Parameter name (use underscores, not hyphens) + type: string # Required: Parameter type + help: Description # Required: Parameter description + required: true # Optional: Whether the parameter is required + default: value # Optional: Default value + choices: [a, b, c] # Optional: For choice/choiceList types + +# Execution Configuration +command: # Either command: or shell-script: is required + - executable # List of command parts - arg1 - "{{ .Args.flag_name }}" + # OR -shell-script: | + +shell-script: | # For complex shell scripts #!/bin/bash echo "Using {{ .Args.flag_name }}" -environment: - ENV_VAR: "{{ .Args.some_flag }}" -cwd: /path/to/working/dir -capture-stderr: true -``` - -IMPORTANT: flag names should use "_", not "-"! - -## Command Types - -### Command Lists - -Command lists are useful for simple commands with fixed arguments: - -```yaml -name: aws-s3-upload -short: Upload a file to S3 -flags: - - name: file - type: string - help: File to upload - required: true - - name: bucket - type: string - help: Target bucket - required: true -command: - - aws - - s3 - - cp - - "{{ .Args.file }}" - - "s3://{{ .Args.bucket }}/" -``` -### Shell Scripts - -Shell scripts are better for complex operations requiring logic: - -```yaml -name: process-logs -short: Process log files -flags: - - name: pattern - type: string - help: Search pattern - required: true - - name: output - type: string - help: Output file - default: output.log -shell-script: | - #!/bin/bash - set -euo pipefail - - echo "Searching for {{ .Args.pattern }}" - find . -type f -name "*.log" -exec grep -H "{{ .Args.pattern }}" {} \; > {{ .Args.output }} +# Optional Configuration +environment: # Optional: Environment variables + ENV_VAR: "{{ .Args.flag_name }}" +cwd: /path/to/working/dir # Optional: Working directory +capture-stderr: true # Optional: Capture stderr in output ``` -## Parameters +## Parameter Types -### Flags - -Flags are named parameters with types and options: - -```yaml -flags: - - name: verbose - type: bool - help: Enable verbose output - default: false - - - name: count - type: int - help: Number of iterations - required: true - - - name: mode - type: choice - help: Operation mode - choices: [fast, safe, debug] - default: safe -``` +The following parameter types are supported: -Supported flag types: +### Basic Types - `string`: Text values (e.g., `--name=value`) - `int`: Integer numbers (e.g., `--count=10`) - `float`: Floating point numbers (e.g., `--threshold=0.75`) - `bool`: True/false values (e.g., `--verbose` or `--verbose=false`) - `date`: Date values (e.g., `--from=2024-01-01`) + +### List Types - `stringList`: List of strings (e.g., `--tags=a,b,c` or multiple `--tags=a --tags=b`) - `intList`: List of integers (e.g., `--numbers=1,2,3`) - `floatList`: List of floating point numbers + +### Choice Types - `choice`: Single selection from predefined options - `choiceList`: Multiple selections from predefined options -- `keyValue`: Key-value pairs (e.g., `--header='Content-Type:application/json'`) + +### File Types - `file`: Single file input - `fileList`: Multiple file inputs - `stringFromFile`: String content read from a file @@ -145,117 +78,64 @@ Supported flag types: - `stringListFromFile`: List of strings read from a file - `objectListFromFile`: List of structured data read from a file -Each flag definition supports these fields: -- `name`: (required) The parameter name used in CLI -- `type`: (required) One of the types listed above -- `help`: Short description of the parameter -- `default`: Default value if not provided -- `required`: Set to true if the parameter must be provided -- `choices`: List of valid options (for `choice` and `choiceList` types) - -Examples of different flag types: - -```yaml -flags: - # Date range with defaults - - name: from - type: date - help: Start date (inclusive) - default: 2024-01-01 - - # Choice from predefined options - - name: group_by - type: choice - help: Result grouping - choices: [year, month, all-time] - default: month - - # List of allowed statuses - - name: status - type: stringList - help: Order statuses to include - default: ['pending', 'processing'] - - # File input - - name: config - type: file - help: Configuration file - required: true - - # Key-value pairs - - name: labels - type: keyValue - help: Resource labels - default: - env: dev - team: backend -``` - -### Arguments +### Special Types +- `keyValue`: Key-value pairs (e.g., `--header='Content-Type:application/json'`) -Arguments are positional parameters: +## Templating System -```yaml -arguments: - - name: source - type: string - help: Source file - required: true - - - name: destination - type: string - help: Destination path -``` +Shell commands use Go's template language for variable interpolation and control flow. Additionally, all [Sprig template functions](http://masterminds.github.io/sprig/) are available for use. -## Environment Variables +### Variable Access -Environment variables can be templated using flag values: +Access flag values using the `.Args` object: ```yaml -flags: - - name: environment - type: string - help: Deployment environment - choices: [dev, staging, prod] - default: dev - -environment: - NODE_ENV: "{{ .Args.environment }}" - DB_HOST: "db.{{ .Args.environment }}.internal" - LOG_LEVEL: "{{ if eq .Args.environment \"prod\" }}error{{ else }}debug{{ end }}" +command: + - echo + - "{{ .Args.name }}" # Access a flag value ``` -## Working Directory +### Control Flow -Set the working directory for command execution: +Use Go template syntax for control flow: ```yaml -name: build -short: Build project -cwd: /path/to/project command: - - make - - build + - rsync + - "{{ if .Args.verbose }}-v{{ end }}" # Conditional + - "{{ range .Args.files }}{{ . }} {{ end }}" # Iteration ``` -## Output Handling +### Sprig Functions -Control stderr capture: +Examples of using Sprig functions: ```yaml -name: risky-operation -short: Run risky operation -capture-stderr: true # Capture stderr in command output shell-script: | #!/bin/bash - set -e - if ! some_command; then - echo "Failed!" >&2 - exit 1 - fi + # String manipulation + NAME="{{ .Args.name | lower | trim }}" + + # Date formatting + DATE="{{ now | date "2006-01-02" }}" + + # List operations + ITEMS="{{ .Args.items | join "," }}" + + # Math operations + COUNT="{{ .Args.number | add 1 }}" ``` +Common Sprig functions: +- String: `trim`, `upper`, `lower`, `title`, `indent` +- Lists: `first`, `last`, `join`, `split`, `compact` +- Math: `add`, `mul`, `div`, `mod` +- Date: `now`, `date`, `dateInZone` +- Encoding: `b64enc`, `b64dec`, `toJson`, `fromJson` + +For a complete list of available functions, visit the [Sprig documentation](http://masterminds.github.io/sprig/). + ## Examples ### Simple Command @@ -286,11 +166,6 @@ command: - "{{ .Args.dest }}" ``` -Usage: -```bash -mcp-client run-command copy-file.yaml --source data.txt --dest backup/ --verbose -``` - ### Complex Script A log processing script with multiple options: @@ -312,7 +187,7 @@ flags: help: Process logs since timestamp default: "24h" - name: format - type: string + type: choice help: Output format choices: [text, json, csv] default: text @@ -320,8 +195,8 @@ shell-script: | #!/bin/bash set -euo pipefail - # Convert time period to timestamp - since_time=$(date -d "{{ .Args.since }} ago" +%s) + # Convert time period to timestamp using Sprig's date functions + since_time=$(date -d "{{ .Args.since | duration "s" }} ago" +%s) # Process each log file find {{ .Args.log_dir }} -type f -name "*.log" | while read -r log; do @@ -490,40 +365,7 @@ shell-script: | - Provide progress information - Format output for readability -## Running Commands - -Commands can be run using the `run-command` subcommand: - -```bash -mcp-server run-command path/to/command.yaml [flags] -``` - -Example: -```bash -mcp-server run-command examples/db-backup.yaml \ - --database myapp \ - --bucket backups.example.com \ - --keep-local -``` - -## Debugging Tips - -1. Use `--help` to see available flags: - ```bash - mcp-server run-command example.yaml --help - ``` - -2. Enable verbose output when available: - ```bash - mcp-server run-command example.yaml --verbose - ``` - -3. Check script output: - ```bash - mcp-server run-command example.yaml --debug - ``` - -## Common Issues +## Troubleshooting 1. **Template Errors** - Check flag names match template variables diff --git a/pinocchio/go-go-mcp/create-command.yaml b/pinocchio/go-go-mcp/create-command.yaml new file mode 100644 index 0000000..f588075 --- /dev/null +++ b/pinocchio/go-go-mcp/create-command.yaml @@ -0,0 +1,227 @@ +name: create-command +short: Generate a shell command YAML file +flags: + - name: additional_system + type: stringList + help: Additional system prompt + - name: additional + type: stringList + help: Additional prompt + - name: description + type: string + help: Description of the command to create + required: true + - name: context + type: fileList + help: Additional context + - name: types + type: stringList + help: List of parameter types + default: + - string + - int + - float + - bool + - date + - stringList + - intList + - floatList + - choice + - choiceList + - file + - fileList + - stringFromFile + - objectFromFile + - stringListFromFile + - objectListFromFile + - keyValue + - name: example_name + type: string + help: Name of the example + default: Process logs + - name: example + type: stringFromFile + help: Example of the command + default: | + name: process-logs + short: Process log files with filtering and output formatting + long: | + Process log files with flexible filtering options and multiple output formats. + Supports pattern matching, time-based filtering, and various output formats. + + flags: + - name: log_dir + type: string + help: Directory containing log files + required: true + + - name: pattern + type: string + help: Search pattern + required: true + + - name: since + type: string + help: Process logs since timestamp + default: "24h" + + - name: format + type: choice + help: Output format + choices: [text, json, csv] + default: text + + - name: output + type: string + help: Output file + default: output.log + + - name: verbose + type: bool + help: Enable verbose output + default: false + + shell-script: | + #!/bin/bash + set -euo pipefail + + # Convert time period to timestamp using Sprig's date functions + since_time=$(date -d "{{ .Args.since | duration "s" }} ago" +%s) + + # Process each log file + find {{ .Args.log_dir }} -type f -name "*.log" | while read -r log; do + if [[ $(stat -c %Y "$log") -gt $since_time ]]; then + case {{ .Args.format }} in + json) + grep "{{ .Args.pattern }}" "$log" | jq -R -s 'split("\n")[:-1] | map({line:.})' + ;; + csv) + grep "{{ .Args.pattern }}" "$log" | sed 's/^/$(basename "$log"),/' >> {{ .Args.output }} + ;; + text|*) + grep "{{ .Args.pattern }}" "$log" + ;; + esac + fi + done + +system-prompt: | + You are an expert shell command designer tasked with creating a YAML template for + shell commands that are to be exposed to an LLM for tool calling. + Your goal is to analyze the given input and create a + standardized, well-structured YAML template that follows go-go-mcp's shell command + format and properly describes the command, its arguments and its flags. + + + {{ .additional_system | join "\n" }} + +prompt: | + Here is the description of the command you need to analyze and transform into a shell command YAML template: + + + {{.description}} + + + Before creating the final YAML template, please analyze the input text and show your reasoning process in tags. In your analysis: + 1. List out the key components of the input text + 2. Identify which parts can be parameterized as flags. Try to minimize the amount of flags, because this will be used as an LLM tool call. + For each flag, write a clear description of what it is for. + 3. Determine if a simple command list or shell script is more appropriate + 4. Consider environment variables and working directory requirements + 5. Plan error handling and output capture strategy + 6. Outline the structure of the command or shell script + + After your analysis, create the YAML template according to the following structure: + + ```yaml + # Metadata Section (Required) + name: # Command name (use lowercase and underscores) + short: # One-line description + long: | # Detailed multi-line description, with multiple usage examples + ... + + # Parameter Definition + flags: # Optional: Command parameters + - name: # Parameter name (use underscores, not hyphens) + type: # Parameter type ({{ .types | join ", " }}) + help: # Parameter description + required: # Whether the parameter is required + default: # Default value if not required + choices: # For choice/choiceList types + + # Execution Configuration (One of command: or shell-script: is required) + command: # For simple commands + - executable + - arg1 + - "{{ "{{ .Args.flag_name }}" }}" + + # OR + + shell-script: | # For complex operations + #!/bin/bash + set -euo pipefail # Always include error handling + ... + + # Optional Configuration + environment: # Environment variables + ENV_VAR: "{{ "{{ .Args.flag_name }}" }}" + cwd: # Working directory + capture-stderr: # Whether to capture stderr in output + ``` + + Shell commands in go-go-mcp use Go's template language for variable interpolation and control flow, + with additional functionality provided by Sprig template functions. This allows for: + - Variable interpolation: {{ "{{ .Args.flag_name }}" }} + - Conditional logic: {{ "{{ if .Args.verbose }}-v{{ end }}" }} + - Loops and ranges: {{ "{{ range .Args.items }}{{ . }} {{ end }}" }} + - String manipulation: {{ "{{ .Args.name | lower | trim }}" }} + - Date handling: {{ "{{" }} now | date "2006-01-02" {{ "}}" }} + - List operations: {{ "{{" }} .Args.list | join "," {{ "}}" }} + - Math operations: {{ "{{" }} .Args.number | add 1 {{ "}}" }} + + Guidelines for creating the YAML template: + + 1. File Structure: + - Use clear sections with comments + - Keep related configuration together + - Follow the order: metadata, flags, execution, optional config + + 2. Naming and Style: + - Use lowercase with underscores for all names + - Write clear, concise descriptions + - Add helpful comments for complex parts + + 3. Parameter Types: + - Choose appropriate types for each flag + - Use choice/choiceList for enumerated values + - Consider using file types for input/output + - Add validation through required/default/choices + + 4. Templating: + - Use Go templates for dynamic values: {{ "{{ .Args.name }}" }} + - Add conditionals where needed: {{ "{{ if condition }}{{ end }}" }} + - Use Sprig functions for advanced operations: + * String: trim, upper, lower, title + * Lists: first, last, join, split + * Math: add, mul, div, mod + * Date: now, date, dateInZone + * Encoding: b64enc, b64dec, toJson + + 5. Error Handling: + - Always use set -euo pipefail in shell scripts + - Add input validation + - Include proper error messages + - Consider cleanup on failure + + 6. Security: + - Validate and sanitize inputs + - Use environment variables for secrets + - Consider working directory implications + - Add appropriate permissions checks + + Example shell command: + ```yaml + {{ .example }} + ``` + + Present your final YAML template enclosed in tags. \ No newline at end of file diff --git a/pkg/doc/topics/02-shell-commands.md b/pkg/doc/topics/02-shell-commands.md new file mode 100644 index 0000000..1439592 --- /dev/null +++ b/pkg/doc/topics/02-shell-commands.md @@ -0,0 +1,512 @@ +--- +Title: Shell Commands in Go Go MCP +Slug: shell-commands +Short: Learn how to create and use shell commands in go-go-mcp. +Topics: + - shell + - commands + - tools +Commands: + - run-command + - start +Flags: + - command + - args + - verbose + - debug +IsTopLevel: true +IsTemplate: false +ShowPerDefault: true +SectionType: GeneralTopic +--- + +This tutorial will guide you through creating and using shell commands in go-go-mcp. Shell commands allow you to define executable commands and scripts in YAML files, with support for templated arguments, environment variables, and flexible execution options. + +## Table of Contents + +1. [Getting Started](#getting-started) +2. [Command Structure](#command-structure) +3. [Templating System](#templating-system) +4. [Basic Shell Command](#basic-shell-command) +5. [Advanced Features](#advanced-features) +6. [Real-World Examples](#real-world-examples) +7. [Best Practices](#best-practices) +8. [Troubleshooting](#troubleshooting) + +## Getting Started + +Shell commands in go-go-mcp are defined in YAML files. These files describe how to execute commands, what parameters they accept, and how to handle their output. + +### Prerequisites + +- go-go-mcp installed +- Basic understanding of YAML +- Basic shell scripting knowledge + +### Directory Setup + +Create a directory for your shell commands: + +```bash +mkdir -p tools/shell-commands +cd tools/shell-commands +``` + +## Command Structure + +A shell command YAML file has the following structure: + +```yaml +# Metadata +name: command-name # Required: Command name (use lowercase and underscores) +short: Short description # Required: One-line description +long: | # Optional: Detailed multi-line description + Detailed description that can + span multiple lines + +# Parameter Definition +flags: # Optional: Command parameters + - name: flag_name # Required: Parameter name (use underscores, not hyphens) + type: string # Required: Parameter type + help: Description # Required: Parameter description + required: true # Optional: Whether the parameter is required + default: value # Optional: Default value + choices: [a, b, c] # Optional: For choice/choiceList types + +# Execution Configuration +command: # Either command: or shell-script: is required + - executable # List of command parts + - arg1 + - "{{ .Args.flag_name }}" + +# OR + +shell-script: | # For complex shell scripts + #!/bin/bash + echo "Using {{ .Args.flag_name }}" + +# Optional Configuration +environment: # Optional: Environment variables + ENV_VAR: "{{ .Args.flag_name }}" +cwd: /path/to/working/dir # Optional: Working directory +capture-stderr: true # Optional: Capture stderr in output +``` + +### Parameter Types + +The following parameter types are supported: + +- **Basic Types** + - `string`: Text values + - `int`: Integer numbers + - `float`: Floating point numbers + - `bool`: True/false values + - `date`: Date values + +- **List Types** + - `stringList`: List of strings + - `intList`: List of integers + - `floatList`: List of floating point numbers + +- **Choice Types** + - `choice`: Single selection from predefined options + - `choiceList`: Multiple selections from predefined options + +- **File Types** + - `file`: Single file input + - `fileList`: Multiple file inputs + - `stringFromFile`: String content read from a file + - `objectFromFile`: Structured data read from a file + - `stringListFromFile`: List of strings read from a file + - `objectListFromFile`: List of structured data read from a file + +- **Special Types** + - `keyValue`: Key-value pairs + +## Templating System + +Shell commands use Go's template language for variable interpolation and control flow. Additionally, all [Sprig template functions](http://masterminds.github.io/sprig/) are available for use. + +### Variable Access + +Access flag values using the `.Args` object: + +```yaml +command: + - echo + - "{{ .Args.name }}" # Access a flag value +``` + +### Control Flow + +Use Go template syntax for control flow: + +```yaml +command: + - rsync + - "{{ if .Args.verbose }}-v{{ end }}" # Conditional + - "{{ range .Args.files }}{{ . }} {{ end }}" # Iteration +``` + +### Sprig Functions + +Examples of using Sprig functions: + +```yaml +shell-script: | + #!/bin/bash + + # String manipulation + NAME="{{ .Args.name | lower | trim }}" + + # Date formatting + DATE="{{ now | date "2006-01-02" }}" + + # List operations + ITEMS="{{ .Args.items | join "," }}" + + # Math operations + COUNT="{{ .Args.number | add 1 }}" +``` + +Common Sprig functions: +- String: `trim`, `upper`, `lower`, `title`, `indent` +- Lists: `first`, `last`, `join`, `split`, `compact` +- Math: `add`, `mul`, `div`, `mod` +- Date: `now`, `date`, `dateInZone` +- Encoding: `b64enc`, `b64dec`, `toJson`, `fromJson` + +## Basic Shell Command + +Let's start with a simple example that copies a file: + +```yaml +# tools/shell-commands/copy-file.yaml +name: copy-file +short: Copy a file with progress +flags: + - name: source + type: string + help: Source file + required: true + - name: dest + type: string + help: Destination path + required: true + - name: verbose + type: bool + help: Show progress + default: false +command: + - rsync + - -av + - "{{ if .Args.verbose }}-P{{ end }}" + - "{{ .Args.source }}" + - "{{ .Args.dest }}" +``` + +### Running Your First Command + +1. Save the above YAML as `copy-file.yaml` +2. Run it using go-go-mcp: + +```bash +go-go-mcp run-command tools/shell-commands/copy-file.yaml \ + --source data.txt \ + --dest backup/ \ + --verbose +``` + +### Understanding the Structure + +Let's break down the key components: + +1. **Metadata** + ```yaml + name: copy-file + short: Copy a file with progress + ``` + These fields identify and describe your command. + +2. **Flags** + ```yaml + flags: + - name: source + type: string + help: Source file + required: true + ``` + Define the parameters your command accepts. + +3. **Command Definition** + ```yaml + command: + - rsync + - -av + ``` + Specify the actual command to execute. + +## Advanced Features + +### Using Shell Scripts + +For more complex operations, use shell scripts: + +```yaml +# tools/shell-commands/process-logs.yaml +name: process-logs +short: Process log files +flags: + - name: pattern + type: string + help: Search pattern + required: true + - name: output + type: string + help: Output file + default: output.log +shell-script: | + #!/bin/bash + set -euo pipefail + + echo "Searching for {{ .Args.pattern }}" + find . -type f -name "*.log" -exec grep -H "{{ .Args.pattern }}" {} \; > {{ .Args.output }} +``` + +### Environment Variables + +Add environment variables to your commands: + +```yaml +# tools/shell-commands/db-backup.yaml +name: db-backup +short: Backup database +flags: + - name: database + type: string + help: Database name + required: true + - name: password + type: string + help: Database password + required: true +environment: + PGPASSWORD: "{{ .Args.password }}" + PGDATABASE: "{{ .Args.database }}" +command: + - pg_dump + - -Fc + - "{{ .Args.database }}" +``` + +### Working Directory + +Specify a working directory for your command: + +```yaml +# tools/shell-commands/build.yaml +name: build +short: Build project +cwd: /path/to/project +command: + - make + - build +``` + +## Real-World Examples + +### Docker Management + +```yaml +# tools/shell-commands/docker-manage.yaml +name: docker-manage +short: Manage Docker containers +flags: + - name: action + type: choice + help: Action to perform + choices: [start, stop, restart, logs] + required: true + - name: container + type: string + help: Container name + required: true +shell-script: | + #!/bin/bash + set -euo pipefail + + case {{ .Args.action }} in + start) + docker start {{ .Args.container }} + ;; + stop) + docker stop {{ .Args.container }} + ;; + restart) + docker restart {{ .Args.container }} + ;; + logs) + docker logs -f {{ .Args.container }} + ;; + esac +``` + +### Git Operations + +```yaml +# tools/shell-commands/git-sync.yaml +name: git-sync +short: Sync git repositories +flags: + - name: repos_dir + type: string + help: Directory containing git repos + required: true + - name: branch + type: string + help: Branch to sync + default: main +shell-script: | + #!/bin/bash + set -euo pipefail + + find {{ .Args.repos_dir }} -type d -name ".git" | while read -r gitdir; do + repo=$(dirname "$gitdir") + echo "Syncing $repo..." + cd "$repo" + git fetch origin + git checkout {{ .Args.branch }} + git pull origin {{ .Args.branch }} + done +``` + +## Best Practices + +### 1. Error Handling + +Always include proper error handling in shell scripts: + +```yaml +shell-script: | + #!/bin/bash + set -euo pipefail + + if ! command -v required-tool &> /dev/null; then + echo "Error: required-tool is not installed" >&2 + exit 1 + fi +``` + +### 2. Input Validation + +Validate inputs before using them: + +```yaml +shell-script: | + #!/bin/bash + set -euo pipefail + + if [[ ! -d "{{ .Args.directory }}" ]]; then + echo "Error: Directory does not exist: {{ .Args.directory }}" >&2 + exit 1 + fi +``` + +### 3. Progress Information + +Keep users informed: + +```yaml +shell-script: | + #!/bin/bash + set -euo pipefail + + echo "Starting operation..." + for item in *; do + echo "Processing $item..." + # process item + done + echo "Operation complete" +``` + +### 4. Flag Naming + +Use underscores in flag names, not hyphens: + +```yaml +flags: + - name: source_dir # Good + type: string + - name: source-dir # Bad + type: string +``` + +## Troubleshooting + +### Common Issues + +1. **Template Errors** + ``` + Error: template: command:1: bad character U+002D '-' + ``` + Solution: Use underscores (_) instead of hyphens (-) in flag names. + +2. **Permission Issues** + ``` + Error: permission denied: ./script.sh + ``` + Solution: Ensure your shell script is executable or run with proper permissions. + +3. **Missing Dependencies** + ``` + Error: command not found: required-tool + ``` + Solution: Install required dependencies or add error checking. + +### Debugging Tips + +1. Enable verbose output: + ```bash + go-go-mcp run-command command.yaml --verbose + ``` + +2. Use debug mode: + ```bash + go-go-mcp run-command command.yaml --debug + ``` + +3. Check command help: + ```bash + go-go-mcp run-command command.yaml --help + ``` + +## Integration with Configuration + +Shell commands can be integrated into your go-go-mcp configuration: + +```yaml +# config.yaml +version: "1" +profiles: + default: + tools: + directories: + - path: ./tools/shell-commands + defaults: + default: + verbose: true +``` + +This allows you to: +1. Load all shell commands from a directory +2. Set default parameters +3. Control access through blacklists/whitelists + +## Next Steps + +1. Create your own shell commands +2. Integrate them with your go-go-mcp configuration +3. Build complex automation workflows +4. Share commands with your team + +Remember to check the [Configuration File Tutorial](01-config-file.md) for more information about integrating shell commands into your go-go-mcp setup. \ No newline at end of file diff --git a/ttmp/2025-02-10/changelog.md b/ttmp/2025-02-10/changelog.md index 393a4a7..8ae5cc6 100644 --- a/ttmp/2025-02-10/changelog.md +++ b/ttmp/2025-02-10/changelog.md @@ -1,5 +1,31 @@ # Configuration File Format Design +Added shell command template generator: +- Created create-command.yaml template for generating shell commands +- Added comprehensive example with log processing +- Included detailed guidelines for command structure +- Added support for Go templates and Sprig functions +- Documented best practices and security considerations + +Updated shell commands documentation: +- Restructured documentation to be clearer and more consistent +- Added detailed section about Go templating and Sprig functions +- Improved command structure documentation with better examples +- Added comprehensive parameter type descriptions +- Unified format between tutorial and reference documentation + +Added comprehensive shell commands tutorial: +- Created detailed tutorial in pkg/doc/topics/02-shell-commands.md +- Covered basic and advanced shell command usage +- Added real-world examples for Docker and Git operations +- Included best practices and troubleshooting guides +- Documented integration with configuration system + +Added documentation for bridge functionality: +- Added section in README about using go-go-mcp as a bridge between SSE and stdio servers +- Documented bridge command usage with example configuration +- Explained use case for stdio-based client integration with web-based MCP servers + Added detailed YAML configuration format design that allows: - Multiple profiles for different environments - Tool and prompt source configuration with parameter filtering @@ -63,4 +89,19 @@ Added example configuration: - Organized tools into meaningful profiles (default, productivity, research, etc.) - Added parameter management examples for each profile - Demonstrated security features with blacklists/whitelists -- Included real-world use cases for different tool combinations \ No newline at end of file +- Included real-world use cases for different tool combinations + +## Update README with installation instructions and command examples + +Updated the README.md to include comprehensive installation instructions matching our other projects' style, and updated command examples to use the installed binary path instead of relative paths. + +- Added installation instructions for homebrew, apt-get, yum, and go get +- Added link to GitHub releases +- Updated all command examples to use `go-go-mcp` instead of `./go-go-mcp` + +Added help system documentation to README: +- Added section about configuration file features and help access +- Added section about shell commands features and help access +- Added overview of help system with examples +- Documented --example flag for viewing example configurations +- Added references to `help --all` for discovering all topics \ No newline at end of file