Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable CorePack Installing Package Managers from Private Registries #11077

Merged
merged 26 commits into from
Dec 11, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
30d8dd2
add registry for corepack install package managers
kbukum1 Dec 7, 2024
07c385c
add registry helper into imports
kbukum1 Dec 7, 2024
8da6ed9
remove lockfile to find out registry for corepack install
kbukum1 Dec 7, 2024
b4031df
fix parameters in testing
kbukum1 Dec 9, 2024
e78b1ff
fix specs
kbukum1 Dec 9, 2024
05ed720
fixed spec
kbukum1 Dec 9, 2024
5c596bf
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 9, 2024
234c4db
add dependabot yml credentials to check npm registry
kbukum1 Dec 9, 2024
9874d68
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 9, 2024
e870538
move registry_helper into package manager helper
kbukum1 Dec 10, 2024
8b8b130
remove registry_helper
kbukum1 Dec 10, 2024
2e90495
fix reverse proxy check
kbukum1 Dec 10, 2024
56f8c7c
fix yarnrc issue
kbukum1 Dec 10, 2024
ec013e3
improve the parsing
kbukum1 Dec 10, 2024
def21df
moved registry helper as seperate class
kbukum1 Dec 10, 2024
0789da6
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 10, 2024
7cd79a6
fix linting issue
kbukum1 Dec 10, 2024
16915d5
fix rubocop checks
kbukum1 Dec 10, 2024
6a0c2ec
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 10, 2024
c97ef73
activate installed package manager
kbukum1 Dec 10, 2024
e745cde
add logging when activating the package manager
kbukum1 Dec 10, 2024
2878c4d
fix spec
kbukum1 Dec 10, 2024
02b9f2f
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 10, 2024
1a3805e
fix logging
kbukum1 Dec 10, 2024
d18afbc
add feature for corepack registry
kbukum1 Dec 11, 2024
679cea2
add feature flag for enable registry for corepack
kbukum1 Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 120 additions & 1 deletion npm_and_yarn/lib/dependabot/npm_and_yarn/package_manager.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
require "dependabot/ecosystem"
require "dependabot/npm_and_yarn/requirement"
require "dependabot/npm_and_yarn/version_selector"
require "dependabot/npm_and_yarn/registry_helper"

module Dependabot
module NpmAndYarn
Expand Down Expand Up @@ -305,6 +304,126 @@ def unsupported?
end
end

class RegistryHelper
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if my previous comment was confusing, I was not referring to the location of the class. What I meant to ask was why this needed to be a standalone class at all?

The class contains two instance variables which are also known by PackageManagerHelper, and all other logic is entirely functional in that there are no changes to stored state. The class is also instantiated in one place only - the constructor of PackageManagerHelper.

The functional nature of the class combined with it's highly scoped usage makes me think that one of two things should be going on here:

  1. If the registry helper is meant to be reusable outside of PackageManagerHelper then it should be instantiated outside of PackageManagerHelper and passed in as an argument. In this way the PackageManagerHelper is entirely agnostic to the idea of registry credentials and configurations

e.g. PackageManagerHelper.new(package_json, lockfiles, registry_helper)

  1. If the RegistryHelper is only ever going to be used within the code of PackageManagerHelper, then RegistryHelper should not be a standalone class. If you want to isolate this logic, you could easily make the class into a module that gets included into PackageManagerHelper

e.g.

module RegistryHelper
  # expect anything including this helper to have accesors for registry_config_files and credentials
  ... 
end

class PackageManagerHelper
  include RegistryHelper
  attr_reader :registry_config_files, :credentials
    
  def initialize(package_json, lockfiles, registry_config_files, credentials)
    ...
    @registry_config_files = registry_config_files
    @credentials = credentials
    ...
  end
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the feedback! The reason I structured it this way is that the long-term goal is to centralize all registry-related logic within the RegistryHelper class. This approach allows us to handle all registry-related operations in one place, making it easier for others to extend or modify the logic in the future, should the need arise for additional registry-related functionality.

By encapsulating the related constants, variables, and logic in a dedicated class, I’m aiming to keep the code clean, modular, and reusable. Using a module would require passing parameters to each function or tightly coupling the logic to PackageManagerHelper, which I believe would reduce flexibility and reusability.

I hope this clarifies my rationale, and I’m happy to discuss further if needed!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can pass registry helper as a object into package manager helper. That can be another way of using it but I am also using similar parameters such as lockfiles. So that's why I initiated the registry helper in the package manager helper.

extend T::Sig

# Keys for configurations
REGISTRY_KEY = "registry"
AUTH_KEY = "authToken"

# Yarn-specific keys
NPM_AUTH_TOKEN_KEY_FOR_YARN = "npmAuthToken"
NPM_SCOPE_KEY_FOR_YARN = "npmScopes"
NPM_REGISTER_KEY_FOR_YARN = "npmRegistryServer"

# Environment variable keys
COREPACK_NPM_REGISTRY_ENV = "COREPACK_NPM_REGISTRY"
COREPACK_NPM_TOKEN_ENV = "COREPACK_NPM_TOKEN"

sig do
params(
registry_config_files: T::Hash[Symbol, T.nilable(Dependabot::DependencyFile)],
credentials: T.nilable(T::Array[Dependabot::Credential])
).void
end
def initialize(registry_config_files, credentials)
@registry_config_files = T.let(registry_config_files, T::Hash[Symbol, T.nilable(Dependabot::DependencyFile)])
@credentials = T.let(credentials, T.nilable(T::Array[Dependabot::Credential]))
end

sig { returns(T::Hash[String, String]) }
def find_corepack_env_variables
registry_info = find_registry_and_token

env_variables = {}
env_variables[COREPACK_NPM_REGISTRY_ENV] = registry_info[:registry] if registry_info[:registry]
env_variables[COREPACK_NPM_TOKEN_ENV] = registry_info[:auth_token] if registry_info[:auth_token]

env_variables
end

private

sig { returns(T::Hash[Symbol, T.nilable(String)]) }
def find_registry_and_token
# Step 1: Check dependabot.yml configuration
dependabot_config = config_npm_registry_and_token
return dependabot_config if dependabot_config[:registry]

# Step 2: Check .npmrc
npmrc_config = @registry_config_files[:npmrc]
npmrc_result = parse_registry_file(npmrc_config)

return npmrc_result if npmrc_result[:registry]

# Step 3: Check .yarnrc
yarnrc_config = @registry_config_files[:yarnrc]
yarnrc_result = parse_registry_file(yarnrc_config)
return yarnrc_result if yarnrc_result[:registry]

# Step 4: Check yarnrc.yml
yarnrc_yml_config = @registry_config_files[:yarnrc_yml]
yarnrc_yml_result = parse_yarnrc_yml(yarnrc_yml_config)
return yarnrc_yml_result if yarnrc_yml_result[:registry]

# Default values if no registry is found
{}
end

sig { returns(T::Hash[Symbol, T.nilable(String)]) }
def config_npm_registry_and_token
registries = {}

return registries unless @credentials&.any?

@credentials.each do |cred|
next unless cred["type"] == "npm_registry"

# Set the registry if it's not already set
registries[:registry] ||= cred["registry"]

# Set the token if it's not already set
registries[:auth_token] ||= cred["token"]
end
registries
end

sig { params(file: T.nilable(Dependabot::DependencyFile)).returns(T::Hash[Symbol, T.nilable(String)]) }
def parse_registry_file(file)
content = file&.content
return {} unless content

result = {}
content.split("\n").each do |line|
key, value = line.split("=", 2)
next unless key && value

result[:registry] = value.strip if key.strip == REGISTRY_KEY
result[:auth_token] = value.strip if key.strip == "_#{AUTH_KEY}"
end
result
end

sig { params(file: T.nilable(Dependabot::DependencyFile)).returns(T::Hash[Symbol, T.nilable(String)]) }
def parse_yarnrc_yml(file)
content = file&.content
return {} unless content

result = {}
yaml_data = YAML.safe_load(content, permitted_classes: [Symbol, String]) || {}
result[:registry] = yaml_data[NPM_REGISTER_KEY_FOR_YARN] if yaml_data.key?(NPM_REGISTER_KEY_FOR_YARN)
result[:auth_token] = yaml_data[NPM_AUTH_TOKEN_KEY_FOR_YARN] if yaml_data.key?(NPM_AUTH_TOKEN_KEY_FOR_YARN)

if yaml_data.key?(NPM_SCOPE_KEY_FOR_YARN)
yaml_data[NPM_SCOPE_KEY_FOR_YARN].each do |_scope, config|
result[:registry] ||= config[NPM_REGISTER_KEY_FOR_YARN]
result[:auth_token] ||= config[NPM_AUTH_TOKEN_KEY_FOR_YARN]
end
end
result
end
end

class PackageManagerHelper
extend T::Sig
extend T::Helpers
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# frozen_string_literal: true

require "dependabot/dependency_file"
require "dependabot/npm_and_yarn/registry_helper"
require "dependabot/npm_and_yarn/helpers"
require "spec_helper"

RSpec.describe Dependabot::NpmAndYarn::RegistryHelper do
Expand Down
Loading