Add new interface based downloader package #1077
Conversation
6cc237b to
f37cee1
Compare
…estability-and' of https://github.com/cloudposse/atmos into feature/dev-3056-refactor-gogetter-utility-for-better-testability-and
|
Caution: we have #1076 we need to merge which modifies the same files |
4639839 to
eafed34
Compare
…estability-and' of https://github.com/cloudposse/atmos into feature/dev-3056-refactor-gogetter-utility-for-better-testability-and
📝 WalkthroughWalkthroughThis pull request introduces a new dependency on Changes
Sequence Diagram(s)sequenceDiagram
participant Config as AtmosConfiguration
participant Downloader as GoGetterDownloader
participant Factory as ClientFactory
participant Client as DownloadClient
Config->>Downloader: NewGoGetterDownloader(atmosConfig)
Downloader->>Factory: NewClient(ctx, src, dest, mode)
Factory-->>Downloader: Return DownloadClient
Downloader->>Client: Fetch / FetchAndAutoParse(src)
Client-->>Downloader: Return download result
Downloader-->>Config: Return processed file content
Suggested labels
Suggested reviewers
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (8)
pkg/downloader/file_downloader_interface.go (2)
20-25: Consider adding context timeout validation.The
NewClientmethod accepts a context but doesn't specify timeout requirements.Add documentation to specify timeout expectations:
// ClientFactory abstracts the creation of a downloader client for better testability +// +// The provided context should include appropriate timeout settings to prevent +// long-running downloads from blocking indefinitely. // //go:generate mockgen -source=$GOFILE -destination=mock_$GOFILE -package=$GOPACKAGE type ClientFactory interface {
32-53: Consider adding String() method for ClientMode.Adding a String() method would improve logging and error messages.
type ClientMode int + +func (m ClientMode) String() string { + switch m { + case ClientModeAny: + return "any" + case ClientModeFile: + return "file" + case ClientModeDir: + return "directory" + default: + return "invalid" + } +}pkg/downloader/file_downloader.go (1)
57-87: Consider adding format-specific error messages.The error messages in
detectFormatAndParsecould be more specific about which format failed to parse.case utils.IsHCL(data): err = hcl.Unmarshal(d, &v) if err != nil { - return nil, err + return nil, fmt.Errorf("failed to parse HCL: %w", err) }pkg/downloader/file_downloader_test.go (2)
12-25: Consider enhancing test coverage with additional assertions.The test verifies basic success scenario but could be more thorough by:
- Asserting that the mock was actually called with expected parameters
- Testing with different timeout values
- Testing with different ClientMode values
41-87: Consider adding negative test cases.While the test covers successful parsing of different formats, it would be beneficial to add test cases for:
- Invalid JSON/YAML/HCL content
- Empty files
- Files with unsupported formats
pkg/downloader/custom_github_detector.go (1)
69-78: Add logging for token injection failures.When token injection is skipped due to existing credentials, consider logging more details to help with debugging.
- u.LogDebug("Credentials found, skipping token injection\n") + u.LogDebug(fmt.Sprintf("Credentials found in URL for %s, skipping token injection\n", parsedURL.Host))internal/exec/vendor_model_component.go (2)
130-130: Consider splitting dependency migration into a separate PR.Based on previous feedback (from PR #768), significant dependency changes like replacing go-getter should be handled in separate PRs to minimize risk and improve reviewability.
181-181: Add error context for download failures.Consider wrapping the error with additional context about the download mode and timeout:
- if err = downloader.NewGoGetterDownloader(atmosConfig).Fetch(p.uri, filepath.Join(tempDir, p.mixinFilename), downloader.ClientModeFile, 10*time.Minute); err != nil { + if err = downloader.NewGoGetterDownloader(atmosConfig).Fetch(p.uri, filepath.Join(tempDir, p.mixinFilename), downloader.ClientModeFile, 10*time.Minute); err != nil { + return fmt.Errorf("failed to download package %s (mode: %v, timeout: %v): %w", p.name, downloader.ClientModeFile, 10*time.Minute, err)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (12)
go.mod(1 hunks)internal/exec/go_getter_utils.go(0 hunks)internal/exec/validate_stacks.go(2 hunks)internal/exec/vendor_model.go(2 hunks)internal/exec/vendor_model_component.go(3 hunks)internal/exec/yaml_func_include.go(2 hunks)pkg/downloader/custom_github_detector.go(1 hunks)pkg/downloader/file_downloader.go(1 hunks)pkg/downloader/file_downloader_interface.go(1 hunks)pkg/downloader/file_downloader_test.go(1 hunks)pkg/downloader/gogetter_downloader.go(1 hunks)pkg/downloader/mock_file_downloader_interface.go(1 hunks)
💤 Files with no reviewable changes (1)
- internal/exec/go_getter_utils.go
🧰 Additional context used
🧠 Learnings (2)
internal/exec/vendor_model_component.go (1)
Learnt from: aknysh
PR: cloudposse/atmos#768
File: internal/exec/vendor_model_component.go:3-20
Timestamp: 2024-11-18T13:59:10.824Z
Learning: When replacing significant dependencies like `go-getter` that require extensive changes, prefer to address them in separate PRs.
internal/exec/validate_stacks.go (1)
Learnt from: haitham911
PR: cloudposse/atmos#731
File: internal/exec/validate_stacks.go:0-0
Timestamp: 2024-11-12T05:52:05.088Z
Learning: In `internal/exec/validate_stacks.go`, when downloading the Atmos JSON Schema file to the temp directory, the temporary file is overwritten each time, so explicit removal is not necessary.
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (13)
pkg/downloader/gogetter_downloader.go (3)
10-17: Well-structured client implementation!The
goGetterClientstruct follows good encapsulation principles by wrapping the go-getter client. TheGetmethod provides a clean interface for download operations.
53-56: LGTM - Clean factory method implementation!The
NewGoGetterDownloaderfunction follows the factory pattern well, properly initializing dependencies.
42-51:❓ Verification inconclusive
Consider thread safety for global state modification.
The function modifies the global
getter.Detectorsslice. This could lead to race conditions in concurrent scenarios.
🏁 Script executed:
#!/bin/bash # Search for concurrent usage of getter.Detectors rg -A 2 "getter\.Detectors"Length of output: 389
Global state thread safety concern: Verify concurrent use of
getter.DetectorsThe update to
getter.DetectorsinregisterCustomDetectorscould be problematic if multiple goroutines invoke it concurrently. Although our search only shows this update in one location, please ensure:
- The function is called only once during initialization or is otherwise protected against concurrent access.
- If concurrent access is possible, consider wrapping the update in a synchronization mechanism (e.g., using a mutex) to avoid potential race conditions.
internal/exec/yaml_func_include.go (1)
51-51: Clean integration of the new downloader!The change effectively integrates the new
FileDownloaderinterface while maintaining the existing error handling pattern.pkg/downloader/file_downloader_interface.go (1)
8-18: Excellent interface design with clear documentation!The
FileDownloaderinterface is well-documented and follows the interface segregation principle.pkg/downloader/file_downloader.go (1)
17-32: Excellent use of dependency injection!The struct design with injected dependencies makes the code highly testable. The use of functional options for tempPathGenerator and fileReader is a great pattern.
pkg/downloader/file_downloader_test.go (1)
27-39: LGTM! Well-structured error handling test.The test properly verifies error propagation and includes appropriate assertions.
pkg/downloader/mock_file_downloader_interface.go (1)
1-141: LGTM! Auto-generated mocks look correct.The mock implementations are properly generated and align with the interfaces.
internal/exec/vendor_model.go (2)
18-18: LGTM! Import of the new downloader package.The addition of the downloader package aligns with the PR's objective to improve testability and abstraction.
272-272: LGTM! Refactored download implementation.The change to use
downloader.NewGoGetterDownloaderimproves testability by abstracting the download functionality into a dedicated package. The timeout of 10 minutes and mode settings are preserved, maintaining the existing behavior.internal/exec/validate_stacks.go (2)
18-18: LGTM! Import of the new downloader package.The addition of the downloader package aligns with the PR's objective to improve testability and abstraction.
404-404: LGTM! Refactored schema download implementation.The change to use
downloader.NewGoGetterDownloaderwith a 30-second timeout and file mode is appropriate for schema downloads. Based on past learnings, the temporary file handling is correct as it gets overwritten on each download.go.mod (1)
27-27: LGTM! Addition of mock package dependency.The addition of
github.com/golang/mockis appropriate for generating mocks, supporting the PR's goal of improving testability.
…or-better-testability-and
…or-better-testability-and
…or-better-testability-and
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
pkg/downloader/file_downloader_test.go (3)
12-25: Consider parameterizing the timeout value.The test is well-structured, but consider making the timeout value configurable or using a shorter duration for tests. This would help speed up test execution while still validating the functionality.
+const testTimeout = 1 * time.Second func TestFileDownloader_Fetch_Success(t *testing.T) { // ... - err := fd.Fetch("src", "dest", ClientModeFile, 10*time.Second) + err := fd.Fetch("src", "dest", ClientModeFile, testTimeout)
27-39: Use predefined errors instead of dynamic error creation.Following Go best practices and the static analysis suggestion, consider using a predefined error variable instead of creating a new error instance.
+var errClientCreationFailed = errors.New("client creation failed") func TestFileDownloader_Fetch_Failure(t *testing.T) { ctrl := gomock.NewController(t) defer ctrl.Finish() mockFactory := NewMockClientFactory(ctrl) - expectedErr := errors.New("client creation failed") - mockFactory.EXPECT().NewClient(gomock.Any(), "src", "dest", ClientModeFile).Return(nil, expectedErr) + mockFactory.EXPECT().NewClient(gomock.Any(), "src", "dest", ClientModeFile).Return(nil, errClientCreationFailed) fd := NewFileDownloader(mockFactory) err := fd.Fetch("src", "dest", ClientModeFile, 10*time.Second) assert.Error(t, err) - assert.Equal(t, expectedErr, errors.Unwrap(err)) + assert.Equal(t, errClientCreationFailed, errors.Unwrap(err))🧰 Tools
🪛 GitHub Check: golangci
[failure] 32-32: [golangci] pkg/downloader/file_downloader_test.go#L32
do not define dynamic errors, use wrapped static errors instead: "errors.New("client creation failed")" (err113)
41-87: Consider improving the test robustness.While the table-driven tests are well-structured, there are a few potential improvements:
- The hardcoded temp file path
/tmp/testfile.jsonmight not work on all platforms (e.g., Windows).- The test data could include more edge cases.
func TestFileDownloader_FetchAndAutoParse(t *testing.T) { // ... - tempFile := "/tmp/testfile.json" + tempFile := t.TempDir() + "/testfile.json" // ... testCases := []struct { name string fileData string + wantErr bool }{ // ... existing cases ... + { + name: "should handle empty json", + fileData: `{}`, + wantErr: false, + }, + { + name: "should handle invalid json", + fileData: `{invalid}`, + wantErr: true, + }, }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
pkg/downloader/file_downloader_test.go(1 hunks)
🧰 Additional context used
🪛 GitHub Check: golangci
pkg/downloader/file_downloader_test.go
[failure] 32-32: [golangci] pkg/downloader/file_downloader_test.go#L32
do not define dynamic errors, use wrapped static errors instead: "errors.New("client creation failed")" (err113)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (1)
pkg/downloader/file_downloader_test.go (1)
1-11: Clean and well-organized imports!The package declaration and imports are well-structured, using appropriate testing utilities.
…or-better-testability-and
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
pkg/downloader/file_downloader_interface.go (2)
8-18: Consider using a more specific return type for FetchAndAutoParse.The
anyreturn type inFetchAndAutoParsecould make the interface harder to use correctly. Consider defining a concrete type or interface that represents the parsed result.- FetchAndAutoParse(src string) (any, error) + FetchAndAutoParse(src string) (ParsedContent, error) +// ParsedContent represents the result of parsing a downloaded file +type ParsedContent interface { + // Add methods that all parsed content must implement +}
27-30: Add error documentation to the Get method.The
Getmethod should document possible error cases to help implementers and users handle errors appropriately.// DownloadClient defines an interface for download operations. type DownloadClient interface { + // Get performs the download operation. + // Returns an error if: + // - the source is unreachable + // - the destination is not writable + // - the download is interrupted Get() error }pkg/downloader/gogetter_downloader.go (1)
34-41: Consider exposing advanced client options
For future flexibility, you might allow passing more go-getter fields (e.g.,getter.Getters,getter.Decompressors) to handle specialized download cases.pkg/downloader/file_downloader.go (2)
47-56: Make timeout configurable
Using a fixed 30-second timeout can be restrictive in certain environments. Consider accepting a custom timeout parameter to enhance flexibility.
58-91: Potential for additional file formats
Your format detection covers many use cases. If you need TOML or other file types, consider extending this switch.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
internal/exec/validate_stacks.go(2 hunks)internal/exec/vendor_model.go(2 hunks)internal/exec/vendor_model_component.go(3 hunks)internal/exec/yaml_func_include.go(2 hunks)pkg/downloader/custom_github_detector.go(1 hunks)pkg/downloader/custom_github_detector_test.go(1 hunks)pkg/downloader/file_downloader.go(1 hunks)pkg/downloader/file_downloader_interface.go(1 hunks)pkg/downloader/file_downloader_test.go(1 hunks)pkg/downloader/gogetter_downloader.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (7)
- internal/exec/vendor_model_component.go
- internal/exec/yaml_func_include.go
- pkg/downloader/file_downloader_test.go
- pkg/downloader/custom_github_detector_test.go
- internal/exec/vendor_model.go
- internal/exec/validate_stacks.go
- pkg/downloader/custom_github_detector.go
🧰 Additional context used
🧠 Learnings (2)
pkg/downloader/gogetter_downloader.go (1)
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/gogetter_downloader.go:19-40
Timestamp: 2025-02-20T18:45:12.549Z
Learning: In the `pkg/downloader` package's `goGetterClientFactory.NewClient` method, explicit validation for `ClientModeInvalid` is not required as the switch statement's default behavior of using `ClientModeAny` is sufficient.
pkg/downloader/file_downloader.go (1)
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/file_downloader.go:34-44
Timestamp: 2025-02-20T18:43:20.587Z
Learning: The `Fetch` method in the `FileDownloader` interface should not clean up downloaded files/folders as they are expected to persist at the specified destination for the caller's use.
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (8)
pkg/downloader/file_downloader_interface.go (3)
1-7: Clean and focused package structure!The package name and imports are well-chosen and minimal.
20-25: Excellent factory pattern implementation!The
ClientFactoryinterface is well-designed with proper context usage and clear parameters.
44-46: Fix documentation capitalization.The documentation for
ClientModeFileandClientModeDirshould start with "Must" to maintain consistency with previous review feedback.- // Be a file path (doesn't have to exist). Src must point to a single + // Must be a file path (doesn't have to exist). The src must point to a single // file. It is saved as dst. ClientModeFile - // A directory path (doesn't have to exist). Src must point to an + // Must be a directory path (doesn't have to exist). The src must point to an // archive or directory (such as in s3). ClientModeDirAlso applies to: 48-50
pkg/downloader/gogetter_downloader.go (3)
14-17: Straightforward delegation to go-getter
This method cleanly defers download execution to the go-getter library, maintaining separation of concerns.
22-32: Well-chosen fallback for client mode
Switching togetter.ClientModeAnywhen no specific mode matches is an effective default that aligns with your learnings. No further validation is necessary.
46-53: Global detectors modification
Appending custom detectors modifies the globalgetter.Detectorsslice. Multiple concurrent calls could race. Check if you need synchronization or a one-time initialization strategy.pkg/downloader/file_downloader.go (2)
25-32: Helpful dependency injection
This factory-based design cleanly injects dependencies. It keeps the implementation testable and modular.
34-45: Robust download logic
Graceful error handling ensures proper client creation. This makes the download process straightforward to maintain.
…estability-and' of https://github.com/cloudposse/atmos into feature/dev-3056-refactor-gogetter-utility-for-better-testability-and
8bd90d2 to
6ea6017
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
pkg/downloader/custom_github_detector.go (3)
14-27: LGTM! Well-structured error definitions and interfaces.The predefined errors and interfaces are well-designed for testing and flexibility. Consider adding error documentation to help API consumers.
var ( + // ErrInvalidGitHubURL is returned when the URL doesn't follow the expected GitHub format ErrInvalidGitHubURL = errors.New("invalid GitHub URL") + // ErrURLEmpty is returned when an empty URL is provided ErrURLEmpty = errors.New("source URL is empty") + // ErrFailedParseURL is returned when URL parsing fails ErrFailedParseURL = errors.New("failed to parse URL") )
54-58: Use predefined error for consistency.The error handling should use the predefined
ErrFailedParseURLfor consistency.parsedURL, err := url.Parse(src) if err != nil { d.Log.Debug(fmt.Sprintf("Failed to parse URL %q: %v", src, err)) - return "", false, fmt.Errorf("failed to parse URL %q: %w", src, err) + return "", false, fmt.Errorf("%w: %v", ErrFailedParseURL, err) }
50-52: Consider making the default scheme more explicit.The default scheme could be made more explicit and configurable.
+const defaultScheme = "https://" + if !strings.Contains(src, "://") { - src = "https://" + src + src = defaultScheme + src }internal/exec/validate_stacks.go (1)
404-404: Consider extracting magic numbers into constants.The timeout duration of 30 seconds is hardcoded. Consider extracting this into a named constant for better maintainability.
+const defaultSchemaDownloadTimeout = 30 * time.Second + func downloadSchemaFromURL(atmosConfig schema.AtmosConfiguration) (string, error) { // ... - if err = downloader.NewGoGetterDownloader(&atmosConfig).Fetch(manifestURL, atmosManifestJsonSchemaFilePath, downloader.ClientModeFile, 30*time.Second); err != nil { + if err = downloader.NewGoGetterDownloader(&atmosConfig).Fetch(manifestURL, atmosManifestJsonSchemaFilePath, downloader.ClientModeFile, defaultSchemaDownloadTimeout); err != nil {
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
internal/exec/validate_stacks.go(2 hunks)internal/exec/vendor_model.go(2 hunks)internal/exec/vendor_model_component.go(3 hunks)internal/exec/yaml_func_include.go(2 hunks)pkg/downloader/custom_github_detector.go(1 hunks)pkg/downloader/custom_github_detector_test.go(1 hunks)pkg/downloader/file_downloader.go(1 hunks)pkg/downloader/file_downloader_interface.go(1 hunks)pkg/downloader/file_downloader_test.go(1 hunks)pkg/downloader/gogetter_downloader.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (8)
- internal/exec/vendor_model_component.go
- internal/exec/yaml_func_include.go
- internal/exec/vendor_model.go
- pkg/downloader/custom_github_detector_test.go
- pkg/downloader/file_downloader.go
- pkg/downloader/gogetter_downloader.go
- pkg/downloader/file_downloader_test.go
- pkg/downloader/file_downloader_interface.go
🧰 Additional context used
🧠 Learnings (2)
internal/exec/validate_stacks.go (1)
Learnt from: haitham911
PR: cloudposse/atmos#731
File: internal/exec/validate_stacks.go:0-0
Timestamp: 2024-11-12T05:52:05.088Z
Learning: In `internal/exec/validate_stacks.go`, when downloading the Atmos JSON Schema file to the temp directory, the temporary file is overwritten each time, so explicit removal is not necessary.
pkg/downloader/custom_github_detector.go (3)
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector_test.go:0-0
Timestamp: 2025-02-21T20:56:20.761Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log`, not `clog`.
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:0-0
Timestamp: 2025-02-21T20:56:05.539Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log` according to the project's import alias configuration.
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:47-68
Timestamp: 2025-02-20T18:33:16.567Z
Learning: Go doesn't have a built-in secure string type. For sensitive data like tokens, using environment variables (which are protected by the OS) is a common and secure practice.
⏰ Context from checks skipped due to timeout of 90000ms (5)
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Acceptance Tests (macos-latest, macos)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Acceptance Tests (ubuntu-latest, linux)
- GitHub Check: Summary
🔇 Additional comments (5)
pkg/downloader/custom_github_detector.go (3)
1-12: LGTM! Clean package structure and imports.The package structure follows Go best practices, and the import alias for
charmbracelet/logcorrectly follows the project's configuration.
28-42: LGTM! Clean struct design with dependency injection.The struct and constructor follow good practices:
- Clear field definitions
- Dependency injection for testability
- Sensible defaults in constructor
92-110: LGTM! Secure and well-structured token handling.The token handling follows security best practices:
- Clear precedence rules
- Environment variables for secure storage
- Comprehensive debug logging
internal/exec/validate_stacks.go (2)
18-18: LGTM! Clean import addition.The new downloader package import aligns with the PR's objective of moving to an interface-based approach.
383-409:✅ Verification successful
Verify error handling in the new downloader implementation.
The function has been refactored to use the new
FileDownloaderinterface, which is a good improvement for testability. However, let's verify the error handling in the new implementation.
🏁 Script executed:
#!/bin/bash # Description: Check error handling in the new downloader implementation # Search for error handling patterns in the downloader package # Check error handling in the Fetch method ast-grep --pattern 'func (d *GoGetterDownloader) Fetch($_, $_, $_, $_) error { $$$ if err != nil { return $$$ } $$$ }' # Check if the error messages are consistent rg -A 2 'return.*error' pkg/downloader/Length of output: 767
Error handling in the new downloader implementation is solid
I verified that the error handling in the
downloadSchemaFromURLfunction is consistent with our practices. The new implementation correctly wraps errors from theFetchmethod, and our tests (for example, those inpkg/downloader/custom_github_detector_test.go) confirm that error conditions—such as invalid or empty URLs—are handled as expected. No changes are required.
4bf0b6d to
5e9cb00
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
pkg/downloader/custom_github_detector.go (3)
29-35: Consider adding config validation.While the constructor is well-structured, it might benefit from validating the config parameter to prevent nil pointer dereferences.
func NewCustomGitHubDetector(config *schema.AtmosConfiguration) *customGitHubDetector { + if config == nil { + config = &schema.AtmosConfiguration{} + } return &customGitHubDetector{ AtmosConfig: config, GetEnv: os.Getenv, } }
37-83: Consider enhancing URL validation and error handling.The URL detection logic is solid, but could be improved:
- Consider validating the URL scheme after defaulting to "https://"
- The error message on line 50 could be simplified since we have ErrFailedParseURL
if !strings.Contains(src, "://") { src = "https://" + src } + +// Validate URL scheme +if !strings.HasPrefix(src, "https://") && !strings.HasPrefix(src, "http://") { + return "", false, fmt.Errorf("%w: invalid scheme", ErrInvalidGitHubURL) +} parsedURL, err := url.Parse(src) if err != nil { log.Debug("Failed to parse URL", "source", src, "error", err) - return "", false, fmt.Errorf("failed to parse URL %q: %w", src, err) + return "", false, ErrFailedParseURL }
85-103: Consider using constants for environment variables.The token selection logic is clear, but using constants for environment variable names would improve maintainability.
+const ( + AtmosGitHubTokenEnv = "ATMOS_GITHUB_TOKEN" + GitHubTokenEnv = "GITHUB_TOKEN" +) func (d *customGitHubDetector) getGitHubToken() (string, string) { - atmosGitHubToken := d.GetEnv("ATMOS_GITHUB_TOKEN") - gitHubToken := d.GetEnv("GITHUB_TOKEN") + atmosGitHubToken := d.GetEnv(AtmosGitHubTokenEnv) + gitHubToken := d.GetEnv(GitHubTokenEnv) if atmosGitHubToken != "" { log.Debug("Using ATMOS_GITHUB_TOKEN") - return atmosGitHubToken, "ATMOS_GITHUB_TOKEN" + return atmosGitHubToken, AtmosGitHubTokenEnv } if d.AtmosConfig.Settings.InjectGithubToken && gitHubToken != "" { log.Debug("Using GITHUB_TOKEN (InjectGithubToken=true)") - return gitHubToken, "GITHUB_TOKEN" + return gitHubToken, GitHubTokenEnv }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
pkg/downloader/custom_github_detector.go(1 hunks)pkg/downloader/file_downloader_interface.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- pkg/downloader/file_downloader_interface.go
🧰 Additional context used
🧠 Learnings (1)
pkg/downloader/custom_github_detector.go (3)
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector_test.go:0-0
Timestamp: 2025-02-21T20:56:20.761Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log`, not `clog`.
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:0-0
Timestamp: 2025-02-21T20:56:05.539Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log` according to the project's import alias configuration.
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:47-68
Timestamp: 2025-02-20T18:33:16.567Z
Learning: Go doesn't have a built-in secure string type. For sensitive data like tokens, using environment variables (which are protected by the OS) is a common and secure practice.
🪛 golangci-lint (1.62.2)
pkg/downloader/custom_github_detector.go
1-1: : # github.com/cloudposse/atmos/pkg/downloader [github.com/cloudposse/atmos/pkg/downloader.test]
pkg/downloader/custom_github_detector_test.go:32:4: unknown field Log in struct literal of type customGitHubDetector
pkg/downloader/custom_github_detector_test.go:55:4: unknown field Log in struct literal of type customGitHubDetector
pkg/downloader/custom_github_detector_test.go:78:4: unknown field Log in struct literal of type customGitHubDetector
pkg/downloader/custom_github_detector_test.go:91:4: unknown field Log in struct literal of type customGitHubDetector
pkg/downloader/custom_github_detector_test.go:104:4: unknown field Log in struct literal of type customGitHubDetector
pkg/downloader/custom_github_detector_test.go:118:4: unknown field Log in struct literal of type customGitHubDetector
pkg/downloader/custom_github_detector_test.go:131:4: unknown field Log in struct literal of type customGitHubDetector
(typecheck)
🪛 GitHub Check: golangci
pkg/downloader/custom_github_detector.go
[failure] 1-1: [golangci] pkg/downloader/custom_github_detector.go#L1
: # github.com/cloudposse/atmos/pkg/downloader [github.com/cloudposse/atmos/pkg/downloader.test]
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
🔇 Additional comments (3)
pkg/downloader/custom_github_detector.go (3)
3-12: LGTM! Well-organized imports.The imports are properly grouped and follow project conventions, with the correct alias for the log package.
14-19: LGTM! Well-defined error variables.The predefined errors follow Go best practices:
- Clear, descriptive messages
- Exported for reuse
- Statically defined for better performance
21-27: LGTM! Clean type definitions promoting testability.The type definitions show good design:
- Function type for environment variable retrieval enables mocking
- Clear struct fields for configuration and dependencies
5e9cb00 to
063767d
Compare
8fbea3b to
91f528c
Compare
087ee9c to
1df3c7c
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
pkg/downloader/custom_github_detector.go (3)
21-35: Add thread safety documentation.The type definitions and constructor are well-structured with good dependency injection. Consider adding documentation about thread safety, especially since this detector might be used concurrently.
Add this documentation above the struct definition:
// customGitHubDetector detects and modifies GitHub URLs. +// This type is safe for concurrent use by multiple goroutines. type customGitHubDetector struct {
37-83: Enhance URL handling robustness.The URL detection and token injection logic is solid, but consider these improvements:
- Sanitize the URL before parsing to prevent potential injection
- Use more specific error wrapping for better error handling
Consider this enhancement:
func (d *customGitHubDetector) Detect(src, _ string) (string, bool, error) { if src == "" { return "", false, ErrURLEmpty } + // Sanitize URL + src = strings.TrimSpace(src) + if !strings.Contains(src, "://") { src = "https://" + src } parsedURL, err := url.Parse(src) if err != nil { log.Debug("Failed to parse URL", "source", src, "error", err) - return "", false, fmt.Errorf("failed to parse URL %q: %w", src, err) + return "", false, fmt.Errorf("%w: %v", ErrFailedParseURL, err) }
85-103: Consider adding token validation.The token selection logic is clear and follows configuration settings. Consider adding basic token validation to catch obviously invalid tokens early.
Add validation like this:
func (d *customGitHubDetector) getGitHubToken() (string, string) { atmosGitHubToken := d.GetEnv("ATMOS_GITHUB_TOKEN") gitHubToken := d.GetEnv("GITHUB_TOKEN") + // Basic token validation + isValidToken := func(token string) bool { + return len(token) > 0 && !strings.Contains(token, " ") + } + if atmosGitHubToken != "" { + if !isValidToken(atmosGitHubToken) { + log.Debug("Invalid ATMOS_GITHUB_TOKEN format") + return "", "" + } log.Debug("Using ATMOS_GITHUB_TOKEN") return atmosGitHubToken, "ATMOS_GITHUB_TOKEN" }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
pkg/downloader/custom_github_detector.go(1 hunks)pkg/downloader/custom_github_detector_test.go(1 hunks)pkg/downloader/file_downloader_interface.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- pkg/downloader/file_downloader_interface.go
- pkg/downloader/custom_github_detector_test.go
🧰 Additional context used
🧠 Learnings (1)
pkg/downloader/custom_github_detector.go (3)
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector_test.go:0-0
Timestamp: 2025-02-21T20:56:20.761Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log`, not `clog`.
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:0-0
Timestamp: 2025-02-21T20:56:05.539Z
Learning: The `github.com/charmbracelet/log` package should be imported with the alias `log` according to the project's import alias configuration.
Learnt from: samtholiya
PR: cloudposse/atmos#1077
File: pkg/downloader/custom_github_detector.go:47-68
Timestamp: 2025-02-20T18:33:16.567Z
Learning: Go doesn't have a built-in secure string type. For sensitive data like tokens, using environment variables (which are protected by the OS) is a common and secure practice.
⏰ Context from checks skipped due to timeout of 90000ms (5)
- GitHub Check: Acceptance Tests (macos-latest, macos)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Acceptance Tests (ubuntu-latest, linux)
- GitHub Check: [localstack] demo-localstack
- GitHub Check: Summary
🔇 Additional comments (2)
pkg/downloader/custom_github_detector.go (2)
1-12: LGTM! Clean package structure and imports.The package organization follows Go conventions, and the log package is correctly aliased as required by the project configuration.
14-19: Well-structured error definitions!Good use of predefined errors with clear naming and descriptive messages. This approach improves error handling and static analysis capabilities.
what
This PR refactors
GoGetterGetandDownloadDetectFormatAndParseFileto improve testability and abstraction. Previously, these were standalone functions ininternal/exec, making them difficult to mock and limiting flexibility in handling different file download mechanisms.Changes Introduced
FileDownloader)gogetterusage, allowing for easy mocking in unit tests.GoGetterGetandDownloadDetectFormatAndParseFileinto a struct implementingFileDownloader.pkg/downloader.why
internal/exec.references
Summary by CodeRabbit
New Features
customGitHubDetectorfor improved handling of GitHub URLs.Refactor
Chores
FileDownloaderfunctionality.customGitHubDetectorfunctionality.